-- Logs begin at Mon 2022-07-18 09:42:29 UTC, end at Mon 2022-07-18 10:45:42 UTC. -- Jul 18 10:14:25.212437 dv-cinder-837546-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 18 10:14:25.219809 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 18 10:14:25.225416 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 18 10:14:25 2022] *** Jul 18 10:14:25.225416 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 18 10:14:25.225416 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 18 10:14:25.225416 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: nodename: dv-cinder-837546-3-iscsi Jul 18 10:14:25.225416 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: machine: x86_64 Jul 18 10:14:25.225416 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: clock source: unix Jul 18 10:14:25.225416 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: pcre jit disabled Jul 18 10:14:25.225416 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: detected number of CPU cores: 4 Jul 18 10:14:25.225416 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: current working directory: / Jul 18 10:14:25.225416 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: detected binary path: /usr/bin/uwsgi-core Jul 18 10:14:25.225416 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: your processes number limit is 47680 Jul 18 10:14:25.225416 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: your memory page size is 4096 bytes Jul 18 10:14:25.225416 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: detected max file descriptor number: 2048 Jul 18 10:14:25.225416 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: lock engine: pthread robust mutexes Jul 18 10:14:25.225416 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: thunder lock: enabled Jul 18 10:14:25.225416 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 18 10:14:25.225416 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 18 10:14:25.264102 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: Python main interpreter initialized at 0x55c9c1ebd9c0 Jul 18 10:14:25.264102 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: python threads support enabled Jul 18 10:14:25.264102 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: your server socket listen backlog is limited to 100 connections Jul 18 10:14:25.264102 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: your mercy for graceful operations on workers is 90 seconds Jul 18 10:14:25.264547 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: mapped 403077 bytes (393 KB) for 2 cores Jul 18 10:14:25.264547 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: *** Operational MODE: preforking *** Jul 18 10:14:25.264858 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: *** uWSGI is running in multiple interpreter mode *** Jul 18 10:14:25.264858 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: spawned uWSGI master process (pid: 80132) Jul 18 10:14:25.265072 dv-cinder-837546-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 18 10:14:25.266204 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: spawned uWSGI worker 1 (pid: 80142, cores: 1) Jul 18 10:14:25.266204 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: spawned uWSGI worker 2 (pid: 80143, cores: 1) Jul 18 10:14:25.266204 dv-cinder-837546-3-iscsi devstack@keystone.service[80132]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 18 10:14:30.733860 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 10:14:30.733860 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: from cryptography.utils import int_from_bytes Jul 18 10:14:30.734662 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 10:14:30.734662 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: from cryptography.utils import int_from_bytes Jul 18 10:14:30.735160 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 10:14:30.735160 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: from cryptography.utils import int_from_bytes Jul 18 10:14:30.735275 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 10:14:30.735275 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: from cryptography.utils import int_from_bytes Jul 18 10:14:30.784011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 10:14:30.784206 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 10:14:30.784206 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 10:14:30.784377 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 10:14:30.784443 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] command line args: [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 10:14:30.784530 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] command line args: [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 10:14:30.784684 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 10:14:30.784766 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 10:14:30.784827 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ================================================================================ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 10:14:30.784891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ================================================================================ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 10:14:30.785086 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] admin_token = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.785186 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] admin_token = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.785331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config_dir = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.785418 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] config_dir = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.785520 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.785602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.785704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] config_source = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.785797 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config_source = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.785878 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] debug = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.785958 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] debug = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.786068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.786383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.786513 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.786623 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.786720 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.786816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.786920 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.787017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.787129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.787257 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.787355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.787451 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.787558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_config_append = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.787668 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_config_append = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.787765 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.787869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.787972 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_dir = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.788082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_dir = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.788191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_file = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.788294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.788415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.788498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.788627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.788716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.788799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.788879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.789001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.789081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.789212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.789291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.789419 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.789515 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.789590 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.789710 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.789787 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.789918 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.789998 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.790116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.790199 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.790313 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.790397 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.790549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.790688 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.790774 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.790853 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.791002 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.791082 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.791256 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.791388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.791550 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.791776 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.791960 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] publish_errors = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.792048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.792197 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.792328 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] publish_errors = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.792414 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.792489 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.792623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.792715 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.792848 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.792928 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.793091 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.793180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.793325 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.793411 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] standard_threads = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.793554 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.793686 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] standard_threads = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.793787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.793867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.794001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.794087 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.794162 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_journal = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.794243 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.794317 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_json = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.794396 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_journal = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.794475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_stderr = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.794555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_json = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.794628 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_syslog = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.794711 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_stderr = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.794826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.794906 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_syslog = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.795075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 10:14:30.795161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.795313 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.795393 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.795562 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.795642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.795804 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.795882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.796054 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.796136 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.796292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.796379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.796561 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.796646 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.796800 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.797144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.797144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.797287 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.797359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.797430 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.797515 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.797596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.797741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.797821 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.797976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.798064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.798211 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.798298 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.798453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.798534 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.798698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.798784 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.798986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.799068 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.799171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.799257 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.799421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.799501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.799668 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.799756 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.799903 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.800041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.800156 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.800259 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.800418 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.800563 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.800696 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.800849 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.800929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.801133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.801227 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.801380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.801468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.801611 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.801693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.801845 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.801937 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.802099 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.802186 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.802344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.802429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.802572 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.802653 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.803328 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.803421 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.803579 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.803659 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.803813 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.803894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.804052 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.804131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.804301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.804388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.804552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.804639 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.804806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.804932 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.805041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.805176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.805335 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.805413 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.805508 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.805631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.805743 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.805862 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.805977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.806093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.806209 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.806340 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.806428 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.806591 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.806681 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.806824 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.806912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.807061 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.807143 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.807290 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.807380 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.807526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.807615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.807762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.807869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.808009 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.808124 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.808223 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.808320 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.808464 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.808598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.808732 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.808834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.808949 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.809081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.809185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.809266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.809418 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.809506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.809647 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.809728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.809878 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.809960 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.810117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.810205 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.810353 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.810435 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.810582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.810715 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.810816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.810906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.811062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.811151 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.811293 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.811376 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.811525 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.811650 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.811758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.811846 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.811995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.812077 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.812253 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.812332 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.812498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.812620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.812770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.812857 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.813006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.813132 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.813243 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.813336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.813482 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.813563 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.813721 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.813809 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.813971 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.814055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.814211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.814293 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.814451 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.814532 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.814697 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.814787 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.814935 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.815016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.815171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.815253 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.815408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.815491 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.815646 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.815727 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.815883 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.815974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.816147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.816231 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.816452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.816583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.816737 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.816818 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.816971 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.817056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.817248 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.817487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.817725 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.817816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.817978 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.818115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.818211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.818428 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.818570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.819232 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.819385 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.819511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.819627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.819785 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.819867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.820059 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.820141 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.820324 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.820414 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.820587 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.820673 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.820842 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.820924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.821081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.821180 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.821342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.821425 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.821603 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.821725 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.821849 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.822001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.822084 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.external = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.822273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.external = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.822354 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.822538 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.822620 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.822815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.822911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.823051 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.password = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.823135 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.823286 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.token = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.823368 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.password = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.823528 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.823640 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.token = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.823767 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.823920 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.824003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.824190 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.824271 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.824475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.824584 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.824756 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.824839 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.824990 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.825106 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.825229 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.825386 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.825468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.825656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.825745 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.825925 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.826015 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.826190 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.826272 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.826439 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.826522 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.826678 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.826769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.826950 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.827033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.827207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.827290 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.827447 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.827538 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.827705 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.827867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.827993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.828158 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.828240 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.828442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.828591 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.828730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.828812 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.829007 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.829099 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.829246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.829330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.829485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.829567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.829735 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.829818 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.829985 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.830069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.830218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.830300 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.830453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.830534 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.830692 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.830774 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.830936 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.831017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.831169 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.831251 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.831417 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.831501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.831654 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.831735 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.831903 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.831986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.832137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.832219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.832384 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.832465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.832656 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.832739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.832891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.832971 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.833121 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.833205 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.833368 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.833448 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.833598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.833680 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.833851 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.833978 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.834099 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.834190 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.834349 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.834432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.834580 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.834660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.834813 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.834906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.835048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.835131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.835294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.835377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.835523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.835612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.835754 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.835837 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.835985 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.836068 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.836229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.836321 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.836478 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.836593 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.836750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.836833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.836999 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.837093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.837233 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.837316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.837482 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.837564 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.837716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.837807 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.837956 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.838039 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.838190 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.838317 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.838428 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.838564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.838655 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.838817 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.838900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.839052 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.839231 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.839380 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.839464 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.839540 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.839711 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.839800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.839942 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.840032 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.840191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.840274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.840439 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.840550 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.840699 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.840795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.840935 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.841016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.841173 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.841255 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.841404 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.841494 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.841640 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.841730 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.841873 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.841963 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.842110 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.842199 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.842346 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.842427 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.842592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.842674 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.842824 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.842913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.843057 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.843138 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.843305 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.843386 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.843552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.843641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.843784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.843867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.844015 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.844097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.844250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.844340 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.844485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.844623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.844745 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.844826 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.844981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.845076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.845216 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.845299 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.845446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.845529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.845686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.845771 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.845945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.846027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.846176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.846261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.846408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.846490 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.846647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.846729 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.846889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.846969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.847126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.847209 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.847355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.847437 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.847589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.847672 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.847824 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.847970 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.848052 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.848224 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.848306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.848476 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.848601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.848761 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.848843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.848995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.849076 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.849240 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.849330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.849477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.849558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.849728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.849811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.849976 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.850071 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.850215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.850298 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.850449 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.850532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.850683 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.850775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.850917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.851257 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.851416 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.851499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.851651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.851741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.851887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.851968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.852122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.852203 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.852359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.852450 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.852620 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.852705 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.852856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.852938 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.853091 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.853187 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.853324 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.853405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.853562 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.853645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.853800 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.853889 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.854033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.854115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.854268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.854350 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.854509 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.854599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.854758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.854841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.855006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.855088 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.855241 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.855331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.855489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.855569 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.855736 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.855817 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.855970 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.856125 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.856204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.856276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.856441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.856595 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.856703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.856798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.856939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] role.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.857023 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.857186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] role.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.857268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.857416 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.857508 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.857665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.857748 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.857914 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.857996 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.858162 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.858253 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.858409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.858498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.858654 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.858737 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.858900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.858983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.859147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.859228 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.859391 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.859475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.859625 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.859708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.859871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.859954 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.860119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.860289 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.860380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.860548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.860635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.860806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.861031 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.861113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.861268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.861353 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.861516 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.861599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.861759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.861840 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.862003 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.862086 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.862252 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.862335 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.862491 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.862572 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.862728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.862812 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.862962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.863044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.863203 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.863283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.863435 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.863518 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.863731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.863813 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.863965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.864056 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.864222 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.864304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.864458 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.864552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.864716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.864810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.864953 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.865036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.865191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.865775 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.865930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.866020 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.866167 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.866250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.866401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.866484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.866649 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.866744 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.866885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.866968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.867129 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.867209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.867374 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.867526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.867608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.867683 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.867839 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.867922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.868076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.868165 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.868313 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.868396 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.868566 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.868652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.868805 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.868901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.869039 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.869122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.869283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.869364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.869527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.869617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.869755 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.869838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.869996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.870089 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.870287 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.870370 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.870523 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.870614 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.870766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.870858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.871004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.871087 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.871235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.871318 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.871469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.871552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.871702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.871783 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.871937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.872020 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.872171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.872255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.872403 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.872486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.872669 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.872754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.872910 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.872995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.873147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.873229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.873379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.873460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.873617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.873700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.873853 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.873934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.874098 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.874179 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.874263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 10:14:30.874341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.874568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.874806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 10:14:30.874975 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 10:14:30.905507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:30.905740 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:30.905856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:30.905969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:30.906090 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:30.906319 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:30.914424 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:30.914684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:30.943135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:30.943341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:30.943468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:30.944140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.088252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.090042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.138356 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.138604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.139133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.139373 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.205146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.205341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.205428 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.205531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.205612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.205698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.205775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.205851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.205999 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.206086 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.206218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.206302 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.232487 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.232747 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.232978 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.233065 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.233205 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.233293 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.233428 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.233527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.233649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.233736 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.233868 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.233955 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.234095 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.234317 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.245227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.245526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.258617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.258849 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.258976 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.259066 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.259213 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.259300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.259439 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.259526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.259661 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.259749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.259879 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.259966 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.260095 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.260312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 10:14:31.277566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.277777 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 10:14:31.278119 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.278224 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.278332 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 10:14:31.278419 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 10:14:31.278603 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.278698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.278867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.278963 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 10:14:31.279128 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.279215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.279385 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.279469 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.279641 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.279898 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.290643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.290923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.291041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.291194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.291321 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.291476 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.291592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.291873 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.299705 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.300022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.303082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.303348 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.303484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.303615 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.303758 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.303905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.304035 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.304280 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.304412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.304563 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.304703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.304840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.304969 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.305100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.305230 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.305370 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.305512 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.305642 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.305773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.305907 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.306045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.306177 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.332856 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.333106 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.334544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.334768 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.334922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.335071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.340167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.340324 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.340451 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.340612 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.340745 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.340875 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.347936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.348070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.348214 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.348342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.353674 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.353820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.353947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.354074 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.354239 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.354370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.354522 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.354650 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.354799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.354933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.355071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.355198 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.355323 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.355449 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.355581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.355707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.374336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.374541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.374669 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.374814 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.379201 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.379460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.383460 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.383689 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.383829 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.383969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.388774 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.389016 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.392352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.392647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.396115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.396405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.396570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.396820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.397455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.397594 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.397725 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.397855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.397985 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.398115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.398264 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.398393 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.417083 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.417428 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.417580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.417736 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.417871 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.418003 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.418135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.418327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.425883 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.426303 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.427596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.427916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.428127 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.428406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.430991 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.431265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.431493 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.431629 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.431766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.431902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.432059 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.432193 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.432347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.432634 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.446012 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.446292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.446572 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.446711 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.446845 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.446989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.447132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.447268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.447402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.447528 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.447663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.447790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.447977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.448252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.467514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.468346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.470957 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.471652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.474276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.475027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.475950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.476996 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.477875 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.478154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.478431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.478654 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.478932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.479219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.485848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.486681 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.487459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.488331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.490853 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.491136 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.491418 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.491645 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.491785 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.491934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.492207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.492481 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.500941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.501212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.501492 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.501865 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.502157 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.502439 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.509856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.510133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.510407 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.510685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.510940 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 10:14:31.511147 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.511346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.511559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.511763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.511969 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.511969 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.512264 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.512408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.512613 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.512751 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.512849 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.512986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.513072 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.513072 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.513211 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.513302 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.513388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.513467 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.513542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.513617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.513854 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.514133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.514395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 10:14:31.551791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c9c1ebd9c0 pid: 80143 (default app) Jul 18 10:14:31.553837 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c9c1ebd9c0 pid: 80142 (default app) Jul 18 10:14:31.555858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6111d-4672-4358-85fd-cb0ec5acdd88 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:31.556150 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6111d-4672-4358-85fd-cb0ec5acdd88 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:31.556437 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6111d-4672-4358-85fd-cb0ec5acdd88 None None] PATH_INFO: `/v3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:31.560734 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Mon Jul 18 10:14:31 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:14:39.421928 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90618a62-9655-4fc4-b276-a5468306aeb2 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:39.422471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90618a62-9655-4fc4-b276-a5468306aeb2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:39.422471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90618a62-9655-4fc4-b276-a5468306aeb2 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:39.426817 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:14:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:14:39.433121 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8410d8b-45d2-4821-b0a6-d850b84b2844 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:39.433617 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8410d8b-45d2-4821-b0a6-d850b84b2844 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:39.433709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8410d8b-45d2-4821-b0a6-d850b84b2844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:39.502447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: INFO dbcounter [None req-c8410d8b-45d2-4821-b0a6-d850b84b2844 None None] Registered counter for database keystone Jul 18 10:14:39.523909 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG oslo_db.sqlalchemy.engines [None req-c8410d8b-45d2-4821-b0a6-d850b84b2844 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80143) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 10:14:39.530356 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writer thread running {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 10:14:39.720111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG passlib.handlers.bcrypt [None req-c8410d8b-45d2-4821-b0a6-d850b84b2844 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80143) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 10:14:39.720449 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG passlib.handlers.bcrypt [None req-c8410d8b-45d2-4821-b0a6-d850b84b2844 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80143) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 10:14:39.779199 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c8410d8b-45d2-4821-b0a6-d850b84b2844 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:14:39.780924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c8410d8b-45d2-4821-b0a6-d850b84b2844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:14:39.911298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:14:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 10:14:40.050038 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90618a62-9655-4fc4-b276-a5468306aeb2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:40.123920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: INFO dbcounter [None req-90618a62-9655-4fc4-b276-a5468306aeb2 None None] Registered counter for database keystone Jul 18 10:14:40.143969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG oslo_db.sqlalchemy.engines [None req-90618a62-9655-4fc4-b276-a5468306aeb2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80142) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 10:14:40.152658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writer thread running {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 10:14:40.272929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90618a62-9655-4fc4-b276-a5468306aeb2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:40.277262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05724993-a394-4dca-a7b9-4045ca069b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:40.277928 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05724993-a394-4dca-a7b9-4045ca069b25 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:40.278221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05724993-a394-4dca-a7b9-4045ca069b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:40.278501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05724993-a394-4dca-a7b9-4045ca069b25 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:40.286967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05724993-a394-4dca-a7b9-4045ca069b25 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:40.339954 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: WARNING keystone.common.rbac_enforcer.enforcer [None req-05724993-a394-4dca-a7b9-4045ca069b25 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 10:14:40.340231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05724993-a394-4dca-a7b9-4045ca069b25 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:40.350557 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Mon Jul 18 10:14:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:14:40.356847 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8410d8b-45d2-4821-b0a6-d850b84b2844 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:40.371366 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8410d8b-45d2-4821-b0a6-d850b84b2844 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:40.375812 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8433e-e3b2-411b-b248-b75afad09ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:40.376966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8433e-e3b2-411b-b248-b75afad09ebb admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:40.377259 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8433e-e3b2-411b-b248-b75afad09ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:40.377539 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8433e-e3b2-411b-b248-b75afad09ebb admin admin] PATH_INFO: `/v3/projects/1282af50d400435ba3f0e00de6e5fb67` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:40.382159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8433e-e3b2-411b-b248-b75afad09ebb admin admin] RBAC: Authorizing `identity:get_project(project_id=1282af50d400435ba3f0e00de6e5fb67)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:40.433182 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7fb8433e-e3b2-411b-b248-b75afad09ebb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 10:14:40.433753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8433e-e3b2-411b-b248-b75afad09ebb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:40.438189 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Mon Jul 18 10:14:40 2022] GET /identity/v3/projects/1282af50d400435ba3f0e00de6e5fb67 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:14:42.451130 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9ba16-938b-4ddc-98e7-d5be1c621e8f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:42.451625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9ba16-938b-4ddc-98e7-d5be1c621e8f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:42.451625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9ba16-938b-4ddc-98e7-d5be1c621e8f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:42.455945 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:14:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:14:42.461999 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbc881-f44f-440d-9060-008333759475 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:42.462394 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbc881-f44f-440d-9060-008333759475 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:42.462559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbc881-f44f-440d-9060-008333759475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:42.481270 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b8bbc881-f44f-440d-9060-008333759475 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:14:42.482801 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b8bbc881-f44f-440d-9060-008333759475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:14:42.490681 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:14:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 10:14:42.512748 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9ba16-938b-4ddc-98e7-d5be1c621e8f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:42.525298 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9ba16-938b-4ddc-98e7-d5be1c621e8f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:42.529518 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4f24b-c103-4ec4-9f20-7cccc3212e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:42.530148 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4f24b-c103-4ec4-9f20-7cccc3212e44 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:42.530442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4f24b-c103-4ec4-9f20-7cccc3212e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:42.530726 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4f24b-c103-4ec4-9f20-7cccc3212e44 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:42.538456 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4f24b-c103-4ec4-9f20-7cccc3212e44 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:42.540238 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4f24b-c103-4ec4-9f20-7cccc3212e44 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:42.549049 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Mon Jul 18 10:14:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:14:42.554222 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbc881-f44f-440d-9060-008333759475 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:42.563122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbc881-f44f-440d-9060-008333759475 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:42.567475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720f0bc-a2d6-474b-b528-a10e35b7921b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:42.568323 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720f0bc-a2d6-474b-b528-a10e35b7921b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:42.568663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720f0bc-a2d6-474b-b528-a10e35b7921b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:42.569446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720f0bc-a2d6-474b-b528-a10e35b7921b admin admin] PATH_INFO: `/v3/users/06ec5a62dde8447fa4c9a00a0f1828f8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:42.570835 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720f0bc-a2d6-474b-b528-a10e35b7921b admin admin] RBAC: Authorizing `identity:get_user(user_id=06ec5a62dde8447fa4c9a00a0f1828f8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:42.572701 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720f0bc-a2d6-474b-b528-a10e35b7921b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:42.574264 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Mon Jul 18 10:14:42 2022] GET /identity/v3/users/06ec5a62dde8447fa4c9a00a0f1828f8 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:14:44.633873 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7619c-c9b2-4a6f-a611-adfe4756943c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:44.634394 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7619c-c9b2-4a6f-a611-adfe4756943c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:44.634488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7619c-c9b2-4a6f-a611-adfe4756943c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:44.635803 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:14:44.641024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9c27a-4b63-40a2-a241-128cd8a65068 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:44.641412 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9c27a-4b63-40a2-a241-128cd8a65068 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:44.641594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9c27a-4b63-40a2-a241-128cd8a65068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:44.653706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ce561-563f-4f12-9c82-f76ce6fd29cb None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:44.654004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ce561-563f-4f12-9c82-f76ce6fd29cb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:44.654291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ce561-563f-4f12-9c82-f76ce6fd29cb None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:44.655664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:14:44.658158 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-27d9c27a-4b63-40a2-a241-128cd8a65068 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:14:44.659679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-27d9c27a-4b63-40a2-a241-128cd8a65068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:14:44.661260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470f24a-9134-4807-b7c2-6f07dd06bedd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:44.661260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470f24a-9134-4807-b7c2-6f07dd06bedd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:44.663345 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470f24a-9134-4807-b7c2-6f07dd06bedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:44.702673 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG passlib.handlers.bcrypt [None req-8470f24a-9134-4807-b7c2-6f07dd06bedd None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80142) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 10:14:44.703100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG passlib.handlers.bcrypt [None req-8470f24a-9134-4807-b7c2-6f07dd06bedd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80142) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 10:14:44.743701 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 6/12] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:14:44.749130 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8470f24a-9134-4807-b7c2-6f07dd06bedd None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:14:44.750743 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8470f24a-9134-4807-b7c2-6f07dd06bedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:14:44.766504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9c27a-4b63-40a2-a241-128cd8a65068 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:44.789961 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:44.849475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 7/13] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:14:44.870466 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9c27a-4b63-40a2-a241-128cd8a65068 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:44.873366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470f24a-9134-4807-b7c2-6f07dd06bedd None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:44.903630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c37771-983a-46f6-9c14-496160d15574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:44.904357 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c37771-983a-46f6-9c14-496160d15574 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:44.904682 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c37771-983a-46f6-9c14-496160d15574 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:44.904964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c37771-983a-46f6-9c14-496160d15574 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:44.916427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c37771-983a-46f6-9c14-496160d15574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:14:44.927653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d7c37771-983a-46f6-9c14-496160d15574 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:14:44.927653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:14:44.927653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:14:44.927653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:14:44.927653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:14:44.927653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:14:44.927653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:14:44.927653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:14:44.927653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:14:44.927653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:14:44.927653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:14:44.927653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:14:44.927653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:14:44.927653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:14:44.927653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:14:44.927653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:14:44.929128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:14:44.929128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:14:44.929128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:14:44.929128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:14:44.929128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:14:44.929128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:14:44.929128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:14:44.929128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:14:44.929128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:14:44.929128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:14:44.929128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:14:44.929128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:14:44.929128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:14:44.929128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:14:44.929128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:14:44.929128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:14:44.929943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:14:44.929943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:14:44.929943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:14:44.929943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:14:44.929943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:14:44.929943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:14:44.929943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:14:44.929943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:14:44.929943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:14:44.929943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:14:44.929943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:14:44.929943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:14:44.929943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:14:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:14:44.935243 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c37771-983a-46f6-9c14-496160d15574 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:44.973347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470f24a-9134-4807-b7c2-6f07dd06bedd None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:44.975708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:45.011635 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8acb52-13b2-4f19-94df-ac6909055d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:45.012596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8acb52-13b2-4f19-94df-ac6909055d63 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:45.012892 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8acb52-13b2-4f19-94df-ac6909055d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:45.013175 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8acb52-13b2-4f19-94df-ac6909055d63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:45.016068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:45.035140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c37771-983a-46f6-9c14-496160d15574 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:45.068470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70071e20-9c8c-46bd-b33d-ea042f31f066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:45.069736 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70071e20-9c8c-46bd-b33d-ea042f31f066 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:45.070024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70071e20-9c8c-46bd-b33d-ea042f31f066 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:45.070304 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70071e20-9c8c-46bd-b33d-ea042f31f066 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:45.071060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70071e20-9c8c-46bd-b33d-ea042f31f066 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:14:45.071376 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70071e20-9c8c-46bd-b33d-ea042f31f066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:45.072807 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70071e20-9c8c-46bd-b33d-ea042f31f066 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:45.081164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8acb52-13b2-4f19-94df-ac6909055d63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:45.082768 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8acb52-13b2-4f19-94df-ac6909055d63 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:45.085001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 8/15] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:14:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:14:45.090322 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70071e20-9c8c-46bd-b33d-ea042f31f066 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:45.180810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 8/16] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:14:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 10:14:45.186425 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8acb52-13b2-4f19-94df-ac6909055d63 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:45.188734 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70071e20-9c8c-46bd-b33d-ea042f31f066 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:45.189179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:45.223570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b537b77-87b2-40f0-8ac3-ab0c674319f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:45.224443 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b537b77-87b2-40f0-8ac3-ab0c674319f5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:45.225788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b537b77-87b2-40f0-8ac3-ab0c674319f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:45.225788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b537b77-87b2-40f0-8ac3-ab0c674319f5 None admin] PATH_INFO: `/v3/users/06ec5a62dde8447fa4c9a00a0f1828f8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:45.227103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b537b77-87b2-40f0-8ac3-ab0c674319f5 None admin] RBAC: Authorizing `identity:get_user(user_id=06ec5a62dde8447fa4c9a00a0f1828f8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:45.228622 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b537b77-87b2-40f0-8ac3-ab0c674319f5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:45.230260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 9/17] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:14:45 2022] GET /identity/v3/users/06ec5a62dde8447fa4c9a00a0f1828f8 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:14:45.235546 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b537b77-87b2-40f0-8ac3-ab0c674319f5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:45.249864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:45.284385 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8acb52-13b2-4f19-94df-ac6909055d63 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:45.318703 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ea543-8eb5-497c-8105-2bcd0ca07d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:45.319515 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ea543-8eb5-497c-8105-2bcd0ca07d98 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:45.319803 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ea543-8eb5-497c-8105-2bcd0ca07d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:45.320082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ea543-8eb5-497c-8105-2bcd0ca07d98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:45.333210 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b537b77-87b2-40f0-8ac3-ab0c674319f5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:45.338504 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ea543-8eb5-497c-8105-2bcd0ca07d98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:45.339976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ea543-8eb5-497c-8105-2bcd0ca07d98 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:45.360805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-975ea543-8eb5-497c-8105-2bcd0ca07d98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:14:45.360805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:14:45.360805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:14:45.360805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:14:45.360805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:14:45.360805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:14:45.360805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:14:45.360805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:14:45.360805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:14:45.360805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:14:45.360805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:14:45.360805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:14:45.360805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:14:45.360805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:14:45.360805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:14:45.361754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:14:45.361754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:14:45.361754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:14:45.361754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:14:45.361754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:14:45.361754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:14:45.361754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:14:45.361754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:14:45.361754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:14:45.361754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:14:45.361754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:14:45.361754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:14:45.361754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:14:45.361754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:14:45.361754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:14:45.361754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:14:45.361754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:14:45.362548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:14:45.362548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:14:45.362548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:14:45.362548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:14:45.362548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 9/18] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:14:45.367722 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05e2ed-ebf7-476d-b174-f981761e9f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:45.367872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ea543-8eb5-497c-8105-2bcd0ca07d98 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:45.368613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05e2ed-ebf7-476d-b174-f981761e9f42 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:45.368841 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05e2ed-ebf7-476d-b174-f981761e9f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:45.369109 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05e2ed-ebf7-476d-b174-f981761e9f42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:45.370294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05e2ed-ebf7-476d-b174-f981761e9f42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:45.371741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05e2ed-ebf7-476d-b174-f981761e9f42 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:45.373396 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 10/19] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:14:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:14:45.378961 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05e2ed-ebf7-476d-b174-f981761e9f42 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:45.426419 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:45.464276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:45.472395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ea543-8eb5-497c-8105-2bcd0ca07d98 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:45.483225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05e2ed-ebf7-476d-b174-f981761e9f42 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:45.506890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a9233-d1cc-4ccc-b5e3-175e5de78c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:45.507553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a9233-d1cc-4ccc-b5e3-175e5de78c63 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:45.507840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a9233-d1cc-4ccc-b5e3-175e5de78c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:45.508124 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a9233-d1cc-4ccc-b5e3-175e5de78c63 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:45.508887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a9233-d1cc-4ccc-b5e3-175e5de78c63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:14:45.509201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a9233-d1cc-4ccc-b5e3-175e5de78c63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:45.510607 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a9233-d1cc-4ccc-b5e3-175e5de78c63 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:45.517578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628678d-43de-49a2-b4da-1456e27ebf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:45.518224 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628678d-43de-49a2-b4da-1456e27ebf0b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:45.518505 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628678d-43de-49a2-b4da-1456e27ebf0b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:45.518778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628678d-43de-49a2-b4da-1456e27ebf0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:45.519607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628678d-43de-49a2-b4da-1456e27ebf0b None admin] RBAC: Adding query filter params (role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.domain.id=default, user.id=06ec5a62dde8447fa4c9a00a0f1828f8) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:14:45.520102 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628678d-43de-49a2-b4da-1456e27ebf0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:45.521367 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628678d-43de-49a2-b4da-1456e27ebf0b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:45.528698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 11/20] 10.222.0.33 () {56 vars in 1322 bytes} [Mon Jul 18 10:14:45 2022] GET /identity/v3/role_assignments?user.id=06ec5a62dde8447fa4c9a00a0f1828f8&scope.domain.id=default&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:14:45.529157 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 10/21] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:14:47.863424 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c9db6-1699-49ed-8b94-0fd0015060c8 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:47.864545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c9db6-1699-49ed-8b94-0fd0015060c8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:47.865135 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c9db6-1699-49ed-8b94-0fd0015060c8 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:47.867050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 12/22] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:14:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:14:47.872321 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3c8ea-03b8-4536-ba67-7770cef5bd21 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:47.872771 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3c8ea-03b8-4536-ba67-7770cef5bd21 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:47.872945 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3c8ea-03b8-4536-ba67-7770cef5bd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:47.896495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-36b3c8ea-03b8-4536-ba67-7770cef5bd21 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:14:47.898386 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-36b3c8ea-03b8-4536-ba67-7770cef5bd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:14:47.979281 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 11/23] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:14:48.013056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c9db6-1699-49ed-8b94-0fd0015060c8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:48.153580 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95f1d4-48c4-4ac5-9a56-8d3c6c6b51b7 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:48.153932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95f1d4-48c4-4ac5-9a56-8d3c6c6b51b7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:48.154148 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95f1d4-48c4-4ac5-9a56-8d3c6c6b51b7 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:48.160553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 12/24] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:14:48.170025 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47529278-3d37-4f99-a13f-eb085e764c32 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:48.170313 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47529278-3d37-4f99-a13f-eb085e764c32 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:48.170596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47529278-3d37-4f99-a13f-eb085e764c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:48.191802 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c9db6-1699-49ed-8b94-0fd0015060c8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:48.208361 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-47529278-3d37-4f99-a13f-eb085e764c32 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:14:48.209634 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-47529278-3d37-4f99-a13f-eb085e764c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:14:48.254010 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a6f4b-884d-40a9-94ae-50f95643788d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:48.254822 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a6f4b-884d-40a9-94ae-50f95643788d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:48.255107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a6f4b-884d-40a9-94ae-50f95643788d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:48.255383 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a6f4b-884d-40a9-94ae-50f95643788d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:48.278151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a6f4b-884d-40a9-94ae-50f95643788d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:14:48.286027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-253a6f4b-884d-40a9-94ae-50f95643788d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:14:48.286027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:14:48.286027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:14:48.286027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:14:48.286027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:14:48.286027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:14:48.286027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:14:48.286027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:14:48.286027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:14:48.286027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:14:48.286027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:14:48.286027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:14:48.286027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:14:48.286027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:14:48.286027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:14:48.286027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:14:48.286889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:14:48.286889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:14:48.286889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:14:48.286889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:14:48.286889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:14:48.286889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:14:48.286889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:14:48.286889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:14:48.286889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:14:48.286889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:14:48.286889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:14:48.286889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:14:48.286889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:14:48.286889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:14:48.286889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:14:48.286889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:14:48.287698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:14:48.287698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:14:48.287698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:14:48.287698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:14:48.287698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:14:48.287698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:14:48.287698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:14:48.287698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:14:48.287698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:14:48.287698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:14:48.287698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:14:48.287698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:14:48.288297 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 13/25] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:14:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:14:48.292538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:48.292701 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e285c-db74-4282-9a41-918df4edb228 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:48.292701 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e285c-db74-4282-9a41-918df4edb228 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:48.292701 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e285c-db74-4282-9a41-918df4edb228 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:48.293202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 14/26] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:14:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:14:48.294053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e285c-db74-4282-9a41-918df4edb228 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:48.297935 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:48.339263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 13/27] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:14:48.340602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e8a4b-b131-458d-a98b-69af5e0b0bf9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:48.340904 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e8a4b-b131-458d-a98b-69af5e0b0bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:48.341188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e8a4b-b131-458d-a98b-69af5e0b0bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:48.384902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d15e8a4b-b131-458d-a98b-69af5e0b0bf9 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:14:48.386174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d15e8a4b-b131-458d-a98b-69af5e0b0bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:14:48.465897 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 14/28] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:14:48.472537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e8a4b-b131-458d-a98b-69af5e0b0bf9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:48.499794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e285c-db74-4282-9a41-918df4edb228 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:48.535206 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147d670-b898-4f4f-8716-3496a0b3dd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:48.536193 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147d670-b898-4f4f-8716-3496a0b3dd9c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:48.536725 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147d670-b898-4f4f-8716-3496a0b3dd9c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:48.537204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147d670-b898-4f4f-8716-3496a0b3dd9c None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:48.538691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d147d670-b898-4f4f-8716-3496a0b3dd9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:14:48.539211 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d147d670-b898-4f4f-8716-3496a0b3dd9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:48.540818 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d147d670-b898-4f4f-8716-3496a0b3dd9c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:48.549461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 15/29] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:14:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:14:48.550946 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f06de6-68e5-4ac4-b7c6-a31eda2835f0 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:48.551435 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f06de6-68e5-4ac4-b7c6-a31eda2835f0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:48.551934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f06de6-68e5-4ac4-b7c6-a31eda2835f0 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:48.554211 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 16/30] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:14:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:14:48.558966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f06de6-68e5-4ac4-b7c6-a31eda2835f0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:48.580147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e8a4b-b131-458d-a98b-69af5e0b0bf9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:48.587683 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:48.616683 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6529e85-3106-4d6a-9755-2551f847a093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:48.617342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6529e85-3106-4d6a-9755-2551f847a093 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:48.617624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6529e85-3106-4d6a-9755-2551f847a093 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:48.617902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6529e85-3106-4d6a-9755-2551f847a093 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:48.628140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6529e85-3106-4d6a-9755-2551f847a093 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:48.629632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6529e85-3106-4d6a-9755-2551f847a093 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:48.631141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:48.647107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b6529e85-3106-4d6a-9755-2551f847a093 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:14:48.647107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:14:48.647107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:14:48.647107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:14:48.647107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:14:48.647107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:14:48.647107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:14:48.647107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:14:48.647107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:14:48.647107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:14:48.647107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:14:48.647107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:14:48.647107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:14:48.647107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:14:48.647107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:14:48.647936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:14:48.647936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:14:48.647936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:14:48.647936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:14:48.647936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:14:48.647936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:14:48.647936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:14:48.647936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:14:48.647936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:14:48.647936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:14:48.647936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:14:48.647936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:14:48.647936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:14:48.647936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:14:48.647936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:14:48.647936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:14:48.647936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:14:48.649126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:14:48.649126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:14:48.649126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:14:48.649126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:14:48.649126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 15/31] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:14:48.653765 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6529e85-3106-4d6a-9755-2551f847a093 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:48.673016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f06de6-68e5-4ac4-b7c6-a31eda2835f0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:48.712099 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6734dd-752b-4058-8982-4b597b66f27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:48.713159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6734dd-752b-4058-8982-4b597b66f27c None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:48.713646 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6734dd-752b-4058-8982-4b597b66f27c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:48.714115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6734dd-752b-4058-8982-4b597b66f27c None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:48.715085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6734dd-752b-4058-8982-4b597b66f27c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:48.716785 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6734dd-752b-4058-8982-4b597b66f27c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:48.766541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6529e85-3106-4d6a-9755-2551f847a093 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:48.805518 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d58820-0eae-433e-993b-623d8f9cda02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:48.806500 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d58820-0eae-433e-993b-623d8f9cda02 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:48.806787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d58820-0eae-433e-993b-623d8f9cda02 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:48.807064 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d58820-0eae-433e-993b-623d8f9cda02 None admin] PATH_INFO: `/v3/users/06ec5a62dde8447fa4c9a00a0f1828f8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:48.808664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d58820-0eae-433e-993b-623d8f9cda02 None admin] RBAC: Authorizing `identity:get_user(user_id=06ec5a62dde8447fa4c9a00a0f1828f8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:48.810202 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d58820-0eae-433e-993b-623d8f9cda02 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:48.811882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 16/32] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:14:48 2022] GET /identity/v3/users/06ec5a62dde8447fa4c9a00a0f1828f8 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:14:48.813106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd8a94-e9d1-4939-b715-128e5350418e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:48.813430 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd8a94-e9d1-4939-b715-128e5350418e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:48.813722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd8a94-e9d1-4939-b715-128e5350418e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:48.832659 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9cbd8a94-e9d1-4939-b715-128e5350418e None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:14:48.833897 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9cbd8a94-e9d1-4939-b715-128e5350418e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:14:48.857472 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:48.884709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 17/33] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Jul 18 10:14:48 2022] POST /identity/v3/roles => generated 219 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:14:48.886402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6734dd-752b-4058-8982-4b597b66f27c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:48.950986 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 17/34] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:14:48.951864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd8a94-e9d1-4939-b715-128e5350418e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:49.038656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6734dd-752b-4058-8982-4b597b66f27c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:49.084303 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:49.104445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1a4ed-8d0b-4c76-b537-17ed5f97b103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:49.107080 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1a4ed-8d0b-4c76-b537-17ed5f97b103 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:49.107535 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1a4ed-8d0b-4c76-b537-17ed5f97b103 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:49.107961 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1a4ed-8d0b-4c76-b537-17ed5f97b103 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:49.108909 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1a4ed-8d0b-4c76-b537-17ed5f97b103 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:14:49.110447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1a4ed-8d0b-4c76-b537-17ed5f97b103 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:49.118072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1a4ed-8d0b-4c76-b537-17ed5f97b103 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:49.145269 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 18/35] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:14:49.146356 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1a4ed-8d0b-4c76-b537-17ed5f97b103 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:49.170467 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd8a94-e9d1-4939-b715-128e5350418e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:49.214267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0150f-380f-421b-ae1d-9010339ffa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:49.215574 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0150f-380f-421b-ae1d-9010339ffa12 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:49.216053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0150f-380f-421b-ae1d-9010339ffa12 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:49.216534 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0150f-380f-421b-ae1d-9010339ffa12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:49.218139 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0150f-380f-421b-ae1d-9010339ffa12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:49.219922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd0150f-380f-421b-ae1d-9010339ffa12 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:49.224101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 18/36] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:14:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:14:49.225763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0150f-380f-421b-ae1d-9010339ffa12 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:49.254087 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:49.267417 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1a4ed-8d0b-4c76-b537-17ed5f97b103 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:49.305302 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438eb980-b409-4fe4-ba42-2ad7afefb7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:49.305900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438eb980-b409-4fe4-ba42-2ad7afefb7f8 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:49.306186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438eb980-b409-4fe4-ba42-2ad7afefb7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:49.306462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438eb980-b409-4fe4-ba42-2ad7afefb7f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:49.307224 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438eb980-b409-4fe4-ba42-2ad7afefb7f8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:49.309278 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438eb980-b409-4fe4-ba42-2ad7afefb7f8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:49.328003 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 19/37] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Jul 18 10:14:49 2022] POST /identity/v3/roles => generated 225 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:14:49.332529 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438eb980-b409-4fe4-ba42-2ad7afefb7f8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:49.367097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0150f-380f-421b-ae1d-9010339ffa12 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:49.396148 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:49.417631 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1c216-abcf-4029-985e-61c649bfb795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:49.418330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1c216-abcf-4029-985e-61c649bfb795 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:49.418621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1c216-abcf-4029-985e-61c649bfb795 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:49.418908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1c216-abcf-4029-985e-61c649bfb795 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:49.419654 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1c216-abcf-4029-985e-61c649bfb795 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:49.421283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1c216-abcf-4029-985e-61c649bfb795 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:49.454768 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438eb980-b409-4fe4-ba42-2ad7afefb7f8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:49.492059 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9583ed7-8ec4-419f-9652-044f6d9cdd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:49.492983 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9583ed7-8ec4-419f-9652-044f6d9cdd53 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:49.493319 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9583ed7-8ec4-419f-9652-044f6d9cdd53 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:49.493608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9583ed7-8ec4-419f-9652-044f6d9cdd53 None admin] PATH_INFO: `/v3/domains/default/users/06ec5a62dde8447fa4c9a00a0f1828f8/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:49.495766 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9583ed7-8ec4-419f-9652-044f6d9cdd53 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=06ec5a62dde8447fa4c9a00a0f1828f8, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:49.497277 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9583ed7-8ec4-419f-9652-044f6d9cdd53 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:49.512583 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 20/38] 10.222.0.33 () {58 vars in 1361 bytes} [Mon Jul 18 10:14:49 2022] PUT /identity/v3/domains/default/users/06ec5a62dde8447fa4c9a00a0f1828f8/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:14:49.519521 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e43596-c855-4d70-baec-8e0af031c64f None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:49.519521 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e43596-c855-4d70-baec-8e0af031c64f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:49.519521 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e43596-c855-4d70-baec-8e0af031c64f None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:49.519521 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 21/39] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:14:49.519521 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad06b3-a96a-402e-9006-827a558ba80f None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:49.519521 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad06b3-a96a-402e-9006-827a558ba80f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:49.519912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad06b3-a96a-402e-9006-827a558ba80f None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:49.519912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 22/40] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:14:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:14:49.525386 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad893c-6790-4db0-a2f7-9c3ee61d67d8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:49.525696 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad893c-6790-4db0-a2f7-9c3ee61d67d8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:49.526835 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad893c-6790-4db0-a2f7-9c3ee61d67d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:49.552106 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-72ad893c-6790-4db0-a2f7-9c3ee61d67d8 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:14:49.553856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-72ad893c-6790-4db0-a2f7-9c3ee61d67d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:14:49.571498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 19/41] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 18 10:14:49 2022] POST /identity/v3/projects => generated 301 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:14:49.573034 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc288300-0c00-4c21-b4e9-1ea83f6d362c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:49.573326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc288300-0c00-4c21-b4e9-1ea83f6d362c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:49.573722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc288300-0c00-4c21-b4e9-1ea83f6d362c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:49.593506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fc288300-0c00-4c21-b4e9-1ea83f6d362c None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:14:49.595571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fc288300-0c00-4c21-b4e9-1ea83f6d362c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:14:49.650543 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 23/42] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:14:49.682284 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad893c-6790-4db0-a2f7-9c3ee61d67d8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:49.715306 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 20/43] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:14:49.730998 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:49.741571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc288300-0c00-4c21-b4e9-1ea83f6d362c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:49.754058 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:49.823536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad893c-6790-4db0-a2f7-9c3ee61d67d8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:49.880442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc288300-0c00-4c21-b4e9-1ea83f6d362c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:49.918918 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac98c2b-3424-4ba0-922d-94ce671c2fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:49.919621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac98c2b-3424-4ba0-922d-94ce671c2fdf None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:49.919912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac98c2b-3424-4ba0-922d-94ce671c2fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:49.920195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac98c2b-3424-4ba0-922d-94ce671c2fdf None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:49.921006 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac98c2b-3424-4ba0-922d-94ce671c2fdf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:49.922578 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac98c2b-3424-4ba0-922d-94ce671c2fdf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:49.938622 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 21/44] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Jul 18 10:14:49 2022] POST /identity/v3/roles => generated 223 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:14:49.976101 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f07851-f71c-41bd-a88a-e38304cbb532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:49.976957 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f07851-f71c-41bd-a88a-e38304cbb532 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:49.977507 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f07851-f71c-41bd-a88a-e38304cbb532 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:49.977790 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f07851-f71c-41bd-a88a-e38304cbb532 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:49.979025 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f07851-f71c-41bd-a88a-e38304cbb532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:49.980539 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f07851-f71c-41bd-a88a-e38304cbb532 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:49.982390 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 24/45] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:14:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:14:49.988087 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac98c2b-3424-4ba0-922d-94ce671c2fdf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:50.063364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:50.105609 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac98c2b-3424-4ba0-922d-94ce671c2fdf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:50.143851 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e192431-0140-4086-83ee-95ac31ce5ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:50.144623 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e192431-0140-4086-83ee-95ac31ce5ff6 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:50.144938 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e192431-0140-4086-83ee-95ac31ce5ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:50.145225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e192431-0140-4086-83ee-95ac31ce5ff6 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:50.145944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e192431-0140-4086-83ee-95ac31ce5ff6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:50.147985 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e192431-0140-4086-83ee-95ac31ce5ff6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:50.175618 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 22/46] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 18 10:14:49 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:14:51.596862 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d81665-28ef-4144-87a4-6cee98951ade None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:51.597348 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d81665-28ef-4144-87a4-6cee98951ade None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:51.597348 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d81665-28ef-4144-87a4-6cee98951ade None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:51.598719 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 25/47] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:14:51.604197 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1fa9d-addd-42bc-a72b-35eeb88ee77a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:51.604606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1fa9d-addd-42bc-a72b-35eeb88ee77a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:51.604917 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1fa9d-addd-42bc-a72b-35eeb88ee77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:51.622403 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f7a1fa9d-addd-42bc-a72b-35eeb88ee77a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:14:51.623614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f7a1fa9d-addd-42bc-a72b-35eeb88ee77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:14:51.798306 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 23/48] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:14:51.822382 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d81665-28ef-4144-87a4-6cee98951ade None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:51.922506 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d81665-28ef-4144-87a4-6cee98951ade None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:51.924888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:51.956129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e836ff9-0ac3-4293-8c95-e6fd8c1fea0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:51.956961 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e836ff9-0ac3-4293-8c95-e6fd8c1fea0c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:51.957251 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e836ff9-0ac3-4293-8c95-e6fd8c1fea0c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:51.957529 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e836ff9-0ac3-4293-8c95-e6fd8c1fea0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:51.967926 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e836ff9-0ac3-4293-8c95-e6fd8c1fea0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:14:51.976094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-2e836ff9-0ac3-4293-8c95-e6fd8c1fea0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:14:51.976094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:14:51.976094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:14:51.976094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:14:51.976094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:14:51.976094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:14:51.976094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:14:51.976094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:14:51.976094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:14:51.976094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:14:51.976094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:14:51.976094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:14:51.976094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:14:51.976094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:14:51.976094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:14:51.976094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:14:51.977041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:14:51.977041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:14:51.977041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:14:51.977041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:14:51.977041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:14:51.977041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:14:51.977041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:14:51.977041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:14:51.977041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:14:51.977041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:14:51.977041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:14:51.977041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:14:51.977041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:14:51.977041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:14:51.977041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:14:51.977041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:14:51.978071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:14:51.978071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:14:51.978071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:14:51.978071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:14:51.978071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:14:51.978071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:14:51.978071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:14:51.978071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:14:51.978071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:14:51.978071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:14:51.978071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:14:51.978071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:14:51.978071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 26/49] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:14:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:14:51.984740 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1fa9d-addd-42bc-a72b-35eeb88ee77a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:52.037390 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:52.090171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1fa9d-addd-42bc-a72b-35eeb88ee77a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:52.123517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de21e68-2254-4962-9abc-d69d6f6aad0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:52.124184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de21e68-2254-4962-9abc-d69d6f6aad0a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:52.124471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de21e68-2254-4962-9abc-d69d6f6aad0a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:52.124780 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de21e68-2254-4962-9abc-d69d6f6aad0a None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:52.125534 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de21e68-2254-4962-9abc-d69d6f6aad0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:14:52.125841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de21e68-2254-4962-9abc-d69d6f6aad0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:52.127224 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de21e68-2254-4962-9abc-d69d6f6aad0a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:52.139237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 24/50] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:14:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:14:52.144802 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e836ff9-0ac3-4293-8c95-e6fd8c1fea0c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:52.188910 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71052a75-935e-40f9-b8ec-576aae330e9c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:52.189139 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71052a75-935e-40f9-b8ec-576aae330e9c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:52.189424 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71052a75-935e-40f9-b8ec-576aae330e9c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:52.190811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 25/51] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:14:52.196135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4450dd-74c2-48f9-ad95-df83e30260f6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:52.196431 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4450dd-74c2-48f9-ad95-df83e30260f6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:52.196747 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4450dd-74c2-48f9-ad95-df83e30260f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:52.213158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8a4450dd-74c2-48f9-ad95-df83e30260f6 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:14:52.214353 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8a4450dd-74c2-48f9-ad95-df83e30260f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:14:52.243504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e836ff9-0ac3-4293-8c95-e6fd8c1fea0c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:52.276974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f55403-9042-49c2-b302-74842dc38e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:52.277841 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f55403-9042-49c2-b302-74842dc38e99 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:52.278135 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f55403-9042-49c2-b302-74842dc38e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:52.278416 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f55403-9042-49c2-b302-74842dc38e99 None admin] PATH_INFO: `/v3/users/06ec5a62dde8447fa4c9a00a0f1828f8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:52.279317 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 26/52] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:14:52.279608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f55403-9042-49c2-b302-74842dc38e99 None admin] RBAC: Authorizing `identity:get_user(user_id=06ec5a62dde8447fa4c9a00a0f1828f8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:52.281179 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f55403-9042-49c2-b302-74842dc38e99 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:52.282816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 27/53] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:14:52 2022] GET /identity/v3/users/06ec5a62dde8447fa4c9a00a0f1828f8 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:14:52.288053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4450dd-74c2-48f9-ad95-df83e30260f6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:52.302088 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f55403-9042-49c2-b302-74842dc38e99 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:52.329464 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:52.334232 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:52.388446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4450dd-74c2-48f9-ad95-df83e30260f6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:52.402709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f55403-9042-49c2-b302-74842dc38e99 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:52.421506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157cdd1d-c1a7-46e9-88e9-0e770ef77059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:52.422293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cdd1d-c1a7-46e9-88e9-0e770ef77059 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:52.422580 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cdd1d-c1a7-46e9-88e9-0e770ef77059 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:52.422869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157cdd1d-c1a7-46e9-88e9-0e770ef77059 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:52.424048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157cdd1d-c1a7-46e9-88e9-0e770ef77059 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:52.425546 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157cdd1d-c1a7-46e9-88e9-0e770ef77059 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:52.427168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 27/54] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:14:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:14:52.432934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157cdd1d-c1a7-46e9-88e9-0e770ef77059 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:52.436557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffd494-33df-48f9-9657-48a953101506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:52.437356 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffd494-33df-48f9-9657-48a953101506 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:52.437646 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffd494-33df-48f9-9657-48a953101506 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:52.437926 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffd494-33df-48f9-9657-48a953101506 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:52.439159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffd494-33df-48f9-9657-48a953101506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:52.440608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffd494-33df-48f9-9657-48a953101506 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:52.442238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 28/55] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:14:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:14:52.448041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffd494-33df-48f9-9657-48a953101506 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:52.531990 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157cdd1d-c1a7-46e9-88e9-0e770ef77059 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:52.540478 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:52.545144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:52.547622 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffd494-33df-48f9-9657-48a953101506 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:52.565381 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2bf0f8-212c-46f5-9273-ef144e838d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:52.566037 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bf0f8-212c-46f5-9273-ef144e838d5f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:52.566330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bf0f8-212c-46f5-9273-ef144e838d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:52.566613 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bf0f8-212c-46f5-9273-ef144e838d5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:52.567458 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2bf0f8-212c-46f5-9273-ef144e838d5f None admin] RBAC: Adding query filter params (role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.domain.id=default, user.id=06ec5a62dde8447fa4c9a00a0f1828f8) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:14:52.567775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2bf0f8-212c-46f5-9273-ef144e838d5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:52.569870 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2bf0f8-212c-46f5-9273-ef144e838d5f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:52.578804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 28/56] 10.222.0.33 () {56 vars in 1322 bytes} [Mon Jul 18 10:14:52 2022] GET /identity/v3/role_assignments?user.id=06ec5a62dde8447fa4c9a00a0f1828f8&scope.domain.id=default&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:14:52.581914 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6216cd3-6aea-4912-8aa2-2dca0a478f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:52.582581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6216cd3-6aea-4912-8aa2-2dca0a478f43 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:52.582868 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6216cd3-6aea-4912-8aa2-2dca0a478f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:52.583147 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6216cd3-6aea-4912-8aa2-2dca0a478f43 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:52.583869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6216cd3-6aea-4912-8aa2-2dca0a478f43 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:52.585424 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6216cd3-6aea-4912-8aa2-2dca0a478f43 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:52.614473 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 29/57] 10.222.0.33 () {60 vars in 1140 bytes} [Mon Jul 18 10:14:52 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:14:54.649855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13347732-27c6-4b86-ae28-e2a8b3be3533 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:54.650362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13347732-27c6-4b86-ae28-e2a8b3be3533 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:54.650459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13347732-27c6-4b86-ae28-e2a8b3be3533 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:54.651815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 29/58] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:14:54.657086 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d620d-f36d-492e-8291-b672cf81363a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:54.657449 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d620d-f36d-492e-8291-b672cf81363a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:54.657660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d620d-f36d-492e-8291-b672cf81363a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:54.674625 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2d3d620d-f36d-492e-8291-b672cf81363a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:14:54.675818 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2d3d620d-f36d-492e-8291-b672cf81363a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:14:54.744936 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 30/59] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:14:54.770091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13347732-27c6-4b86-ae28-e2a8b3be3533 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:54.870451 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13347732-27c6-4b86-ae28-e2a8b3be3533 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:54.903016 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bccd89-91ae-456c-bbd4-ef8e4f67aa22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:54.903741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bccd89-91ae-456c-bbd4-ef8e4f67aa22 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:54.904031 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bccd89-91ae-456c-bbd4-ef8e4f67aa22 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:54.904316 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bccd89-91ae-456c-bbd4-ef8e4f67aa22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:54.905524 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bccd89-91ae-456c-bbd4-ef8e4f67aa22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:54.906982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bccd89-91ae-456c-bbd4-ef8e4f67aa22 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:54.908615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 30/60] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:14:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:14:54.914893 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d620d-f36d-492e-8291-b672cf81363a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:54.992752 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:55.010941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d620d-f36d-492e-8291-b672cf81363a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:55.050813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01fae6-554c-4e2d-85e9-80231bb26df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:55.051510 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01fae6-554c-4e2d-85e9-80231bb26df2 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:55.051797 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01fae6-554c-4e2d-85e9-80231bb26df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:55.052082 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01fae6-554c-4e2d-85e9-80231bb26df2 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:55.052826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01fae6-554c-4e2d-85e9-80231bb26df2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:55.054312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01fae6-554c-4e2d-85e9-80231bb26df2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:55.078109 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 31/61] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 18 10:14:54 2022] POST /identity/v3/users => generated 275 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:14:58.247036 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369eef0d-bb28-4b95-b0c3-99be38aa0e03 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:58.248009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369eef0d-bb28-4b95-b0c3-99be38aa0e03 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:58.248488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369eef0d-bb28-4b95-b0c3-99be38aa0e03 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:58.250320 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 31/62] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:14:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:14:58.255936 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a377a65-d7a2-418c-921c-1164ef1804fc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:58.257321 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a377a65-d7a2-418c-921c-1164ef1804fc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:58.257826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a377a65-d7a2-418c-921c-1164ef1804fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:58.278187 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2a377a65-d7a2-418c-921c-1164ef1804fc None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:14:58.279402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2a377a65-d7a2-418c-921c-1164ef1804fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:14:58.375116 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 32/63] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:14:58.383361 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d122b-58da-443c-bd94-2fa557469c1c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:58.383928 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d122b-58da-443c-bd94-2fa557469c1c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:58.384403 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d122b-58da-443c-bd94-2fa557469c1c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:58.386004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 32/64] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:14:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:14:58.391191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317d8d1-0719-4768-a642-45c4d36e3b50 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:58.391495 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317d8d1-0719-4768-a642-45c4d36e3b50 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:58.391779 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317d8d1-0719-4768-a642-45c4d36e3b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:58.400910 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a20d94-9aec-4322-942e-5ddf98929f0c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:58.401547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a20d94-9aec-4322-942e-5ddf98929f0c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:58.402028 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a20d94-9aec-4322-942e-5ddf98929f0c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:58.403868 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 33/65] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:14:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:14:58.409211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977bc96-1197-4c82-8e86-f537f1c28815 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:58.409677 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977bc96-1197-4c82-8e86-f537f1c28815 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:58.410195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977bc96-1197-4c82-8e86-f537f1c28815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:58.414431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5317d8d1-0719-4768-a642-45c4d36e3b50 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:14:58.415634 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5317d8d1-0719-4768-a642-45c4d36e3b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:14:58.428152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9977bc96-1197-4c82-8e86-f537f1c28815 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:14:58.430215 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9977bc96-1197-4c82-8e86-f537f1c28815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:14:58.468318 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:58.497355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:58.506256 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 33/66] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:14:58.507511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317d8d1-0719-4768-a642-45c4d36e3b50 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:58.516804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 34/67] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:14:58.551457 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977bc96-1197-4c82-8e86-f537f1c28815 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:58.624856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317d8d1-0719-4768-a642-45c4d36e3b50 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:58.662795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fca5b3-7eb3-4067-b2c2-645f7151e410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:58.663552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fca5b3-7eb3-4067-b2c2-645f7151e410 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:58.663841 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fca5b3-7eb3-4067-b2c2-645f7151e410 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:58.664056 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977bc96-1197-4c82-8e86-f537f1c28815 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:58.664153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fca5b3-7eb3-4067-b2c2-645f7151e410 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:58.677269 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fca5b3-7eb3-4067-b2c2-645f7151e410 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:14:58.686098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-50fca5b3-7eb3-4067-b2c2-645f7151e410 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:14:58.686098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:14:58.686098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:14:58.686098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:14:58.686098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:14:58.686098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:14:58.686098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:14:58.686098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:14:58.686098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:14:58.686098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:14:58.686098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:14:58.686098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:14:58.686098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:14:58.686098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:14:58.686098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:14:58.686098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:14:58.686940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:14:58.686940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:14:58.686940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:14:58.686940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:14:58.686940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:14:58.686940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:14:58.686940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:14:58.686940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:14:58.686940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:14:58.686940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:14:58.686940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:14:58.686940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:14:58.686940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:14:58.686940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:14:58.686940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:14:58.686940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:14:58.688084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:14:58.688084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:14:58.688084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:14:58.688084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:14:58.688084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:14:58.688084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:14:58.688084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:14:58.688084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:14:58.688084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:14:58.688084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:14:58.688084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:14:58.688084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:14:58.688084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 34/68] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:14:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:14:58.688084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fca5b3-7eb3-4067-b2c2-645f7151e410 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:58.705166 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf85537-36d4-4a42-8550-3a1f944135fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:58.705983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf85537-36d4-4a42-8550-3a1f944135fe None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:58.706271 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf85537-36d4-4a42-8550-3a1f944135fe None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:58.706548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf85537-36d4-4a42-8550-3a1f944135fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:58.718213 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf85537-36d4-4a42-8550-3a1f944135fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:14:58.727700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ebf85537-36d4-4a42-8550-3a1f944135fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:14:58.727700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:14:58.727700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:14:58.727700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:14:58.727700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:14:58.727700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:14:58.727700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:14:58.727700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:14:58.727700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:14:58.727700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:14:58.727700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:14:58.727700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:14:58.727700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:14:58.727700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:14:58.727700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:14:58.727700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:14:58.728570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:14:58.728570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:14:58.728570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:14:58.728570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:14:58.728570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:14:58.728570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:14:58.728570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:14:58.728570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:14:58.728570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:14:58.728570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:14:58.728570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:14:58.728570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:14:58.728570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:14:58.728570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:14:58.728570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:14:58.728570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:14:58.729351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:14:58.729351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:14:58.729351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:14:58.729351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:14:58.729351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:14:58.729351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:14:58.729351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:14:58.729351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:14:58.729351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:14:58.729351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:14:58.729351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:14:58.729351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:14:58.729351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 35/69] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:14:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:14:58.730492 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1cc2a-84fe-4063-aa50-57d2b5c9e9bb None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:58.730492 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1cc2a-84fe-4063-aa50-57d2b5c9e9bb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:58.730746 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1cc2a-84fe-4063-aa50-57d2b5c9e9bb None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:58.732184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 36/70] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:14:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:14:58.736804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1cc2a-84fe-4063-aa50-57d2b5c9e9bb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:58.757429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:58.769444 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:58.806128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fca5b3-7eb3-4067-b2c2-645f7151e410 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:58.841279 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0813dc-60eb-49a9-8b1a-f369a634ab0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:58.842128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0813dc-60eb-49a9-8b1a-f369a634ab0b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:58.842421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0813dc-60eb-49a9-8b1a-f369a634ab0b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:58.842704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0813dc-60eb-49a9-8b1a-f369a634ab0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:58.844402 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1cc2a-84fe-4063-aa50-57d2b5c9e9bb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:58.853007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0813dc-60eb-49a9-8b1a-f369a634ab0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:14:58.860594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-fe0813dc-60eb-49a9-8b1a-f369a634ab0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:14:58.860594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:14:58.860594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:14:58.860594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:14:58.860594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:14:58.860594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:14:58.860594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:14:58.860594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:14:58.860594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:14:58.860594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:14:58.860594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:14:58.860594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:14:58.860594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:14:58.860594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:14:58.860594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:14:58.860594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:14:58.861514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:14:58.861514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:14:58.861514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:14:58.861514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:14:58.861514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:14:58.861514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:14:58.861514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:14:58.861514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:14:58.861514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:14:58.861514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:14:58.861514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:14:58.861514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:14:58.861514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:14:58.861514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:14:58.861514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:14:58.861514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:14:58.862483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:14:58.862483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:14:58.862483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:14:58.862483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:14:58.862483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:14:58.862483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:14:58.862483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:14:58.862483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:14:58.862483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:14:58.862483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:14:58.862483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:14:58.862483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:14:58.862483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 35/71] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:14:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:14:58.863599 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0813dc-60eb-49a9-8b1a-f369a634ab0b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:58.881146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092cce4-6ce4-4a09-8ff6-7a3a411f5f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:58.882467 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092cce4-6ce4-4a09-8ff6-7a3a411f5f09 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:58.882761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092cce4-6ce4-4a09-8ff6-7a3a411f5f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:58.883039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092cce4-6ce4-4a09-8ff6-7a3a411f5f09 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:58.883791 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092cce4-6ce4-4a09-8ff6-7a3a411f5f09 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:14:58.884257 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092cce4-6ce4-4a09-8ff6-7a3a411f5f09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:58.886189 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092cce4-6ce4-4a09-8ff6-7a3a411f5f09 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:58.893923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 37/72] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:14:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:14:58.894986 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f159cf-1934-4eb7-9243-0cc69281f66a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:58.895277 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f159cf-1934-4eb7-9243-0cc69281f66a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:58.895562 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f159cf-1934-4eb7-9243-0cc69281f66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:58.912218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c5f159cf-1934-4eb7-9243-0cc69281f66a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:14:58.914063 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c5f159cf-1934-4eb7-9243-0cc69281f66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:14:58.966795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0813dc-60eb-49a9-8b1a-f369a634ab0b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:58.980659 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 38/73] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:14:58.981978 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e63cd-95e2-4293-bee4-3aa1eb2a487b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:58.982276 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e63cd-95e2-4293-bee4-3aa1eb2a487b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:58.982562 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e63cd-95e2-4293-bee4-3aa1eb2a487b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:58.983843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 39/74] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:14:58.984608 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e63cd-95e2-4293-bee4-3aa1eb2a487b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:58.998889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:59.014861 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:59.017685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41960cd0-2603-4896-8df3-0aeaf0a488f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:59.018358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41960cd0-2603-4896-8df3-0aeaf0a488f2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:59.018643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41960cd0-2603-4896-8df3-0aeaf0a488f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:59.018927 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41960cd0-2603-4896-8df3-0aeaf0a488f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:59.019683 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41960cd0-2603-4896-8df3-0aeaf0a488f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:14:59.019991 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41960cd0-2603-4896-8df3-0aeaf0a488f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:59.021432 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41960cd0-2603-4896-8df3-0aeaf0a488f2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:59.029999 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 36/75] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:14:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:14:59.030791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41960cd0-2603-4896-8df3-0aeaf0a488f2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:59.092766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e63cd-95e2-4293-bee4-3aa1eb2a487b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:59.125480 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ef05c-e026-47eb-99e5-ed6cbc8abb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:59.126146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ef05c-e026-47eb-99e5-ed6cbc8abb4e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:59.126436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ef05c-e026-47eb-99e5-ed6cbc8abb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:59.126721 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ef05c-e026-47eb-99e5-ed6cbc8abb4e None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:59.126928 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41960cd0-2603-4896-8df3-0aeaf0a488f2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:59.127475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ef05c-e026-47eb-99e5-ed6cbc8abb4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:14:59.127790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ef05c-e026-47eb-99e5-ed6cbc8abb4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:59.129223 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ef05c-e026-47eb-99e5-ed6cbc8abb4e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:59.137895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 40/76] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:14:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:14:59.138873 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e000f-f031-45c6-841c-e38412847c62 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:59.139168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e000f-f031-45c6-841c-e38412847c62 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:59.139452 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e000f-f031-45c6-841c-e38412847c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:59.155595 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d73e000f-f031-45c6-841c-e38412847c62 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:14:59.156820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d73e000f-f031-45c6-841c-e38412847c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:14:59.166634 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef14a0-5780-493f-96d9-397260e1cfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:59.167446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef14a0-5780-493f-96d9-397260e1cfa8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:59.167730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef14a0-5780-493f-96d9-397260e1cfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:59.168005 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef14a0-5780-493f-96d9-397260e1cfa8 None admin] PATH_INFO: `/v3/users/4eecd6829d5a4a319fa4af969eb3d39d` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:59.183259 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef14a0-5780-493f-96d9-397260e1cfa8 None admin] RBAC: Authorizing `identity:get_user(user_id=4eecd6829d5a4a319fa4af969eb3d39d)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:59.184822 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef14a0-5780-493f-96d9-397260e1cfa8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:59.189475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 37/77] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:14:59 2022] GET /identity/v3/users/4eecd6829d5a4a319fa4af969eb3d39d => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:14:59.189475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef14a0-5780-493f-96d9-397260e1cfa8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:59.232260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 41/78] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:14:59.232260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e000f-f031-45c6-841c-e38412847c62 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:59.233203 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:59.246780 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:59.289295 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef14a0-5780-493f-96d9-397260e1cfa8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:59.324088 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe75627-7b7b-443a-b064-724f67f0e498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:59.324914 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe75627-7b7b-443a-b064-724f67f0e498 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:59.325206 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe75627-7b7b-443a-b064-724f67f0e498 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:59.325484 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe75627-7b7b-443a-b064-724f67f0e498 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:59.330115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e000f-f031-45c6-841c-e38412847c62 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:59.335493 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe75627-7b7b-443a-b064-724f67f0e498 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:14:59.343007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0fe75627-7b7b-443a-b064-724f67f0e498 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:14:59.343007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:14:59.343007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:14:59.343007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:14:59.343007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:14:59.343007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:14:59.343007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:14:59.343007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:14:59.343007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:14:59.343007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:14:59.343007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:14:59.343007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:14:59.343007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:14:59.343007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:14:59.343007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:14:59.343007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:14:59.343823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:14:59.343823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:14:59.343823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:14:59.343823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:14:59.343823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:14:59.343823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:14:59.343823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:14:59.343823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:14:59.343823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:14:59.343823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:14:59.343823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:14:59.343823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:14:59.343823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:14:59.343823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:14:59.343823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:14:59.343823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:14:59.349659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:14:59.349659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:14:59.349659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:14:59.349659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:14:59.349659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:14:59.349659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:14:59.349659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:14:59.349659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:14:59.349659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:14:59.349659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:14:59.349659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:14:59.349659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:14:59.349659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 38/79] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:14:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:14:59.349659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe75627-7b7b-443a-b064-724f67f0e498 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:59.364625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f11c6-7ab3-4d47-83ce-dade43e0ec72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:59.365500 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f11c6-7ab3-4d47-83ce-dade43e0ec72 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:59.365791 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f11c6-7ab3-4d47-83ce-dade43e0ec72 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:59.366616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f11c6-7ab3-4d47-83ce-dade43e0ec72 None admin] PATH_INFO: `/v3/users/4eecd6829d5a4a319fa4af969eb3d39d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:59.367825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f11c6-7ab3-4d47-83ce-dade43e0ec72 None admin] RBAC: Authorizing `identity:get_user(user_id=4eecd6829d5a4a319fa4af969eb3d39d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:59.369321 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f11c6-7ab3-4d47-83ce-dade43e0ec72 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:59.370968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 42/80] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:14:59 2022] GET /identity/v3/users/4eecd6829d5a4a319fa4af969eb3d39d => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:14:59.371734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f11c6-7ab3-4d47-83ce-dade43e0ec72 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:59.449127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:59.452656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe75627-7b7b-443a-b064-724f67f0e498 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:59.452941 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:59.472076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f11c6-7ab3-4d47-83ce-dade43e0ec72 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:59.487978 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced2311-44f0-484b-a841-c5c35945a7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:59.488881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced2311-44f0-484b-a841-c5c35945a7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:59.489174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced2311-44f0-484b-a841-c5c35945a7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:59.489462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced2311-44f0-484b-a841-c5c35945a7d1 None admin] PATH_INFO: `/v3/users/06ec5a62dde8447fa4c9a00a0f1828f8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:59.490677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced2311-44f0-484b-a841-c5c35945a7d1 None admin] RBAC: Authorizing `identity:get_user(user_id=06ec5a62dde8447fa4c9a00a0f1828f8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:59.492810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced2311-44f0-484b-a841-c5c35945a7d1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:59.494458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 39/81] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:14:59 2022] GET /identity/v3/users/06ec5a62dde8447fa4c9a00a0f1828f8 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:14:59.495235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced2311-44f0-484b-a841-c5c35945a7d1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:59.510066 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3dfc11-2692-45fc-a754-d8b67d1eb1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:59.510720 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3dfc11-2692-45fc-a754-d8b67d1eb1cc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:59.511007 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3dfc11-2692-45fc-a754-d8b67d1eb1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:59.511285 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3dfc11-2692-45fc-a754-d8b67d1eb1cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:59.576138 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3dfc11-2692-45fc-a754-d8b67d1eb1cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:59.577643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3dfc11-2692-45fc-a754-d8b67d1eb1cc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:59.592681 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced2311-44f0-484b-a841-c5c35945a7d1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:59.626498 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485dc01-4462-438c-b67e-0cbd58861866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:59.627280 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485dc01-4462-438c-b67e-0cbd58861866 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:59.627566 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485dc01-4462-438c-b67e-0cbd58861866 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:59.627842 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485dc01-4462-438c-b67e-0cbd58861866 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:59.629056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485dc01-4462-438c-b67e-0cbd58861866 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:59.630502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485dc01-4462-438c-b67e-0cbd58861866 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:59.632430 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 40/82] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:14:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:14:59.633025 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485dc01-4462-438c-b67e-0cbd58861866 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:59.656737 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:59.667724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:59.675771 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 43/83] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:14:59.676647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3dfc11-2692-45fc-a754-d8b67d1eb1cc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:59.734485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485dc01-4462-438c-b67e-0cbd58861866 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:59.766686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffe0de-9769-4e68-85a4-b687828d6b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:59.767348 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffe0de-9769-4e68-85a4-b687828d6b4c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:59.767636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffe0de-9769-4e68-85a4-b687828d6b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:59.767912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffe0de-9769-4e68-85a4-b687828d6b4c None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:59.768690 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffe0de-9769-4e68-85a4-b687828d6b4c None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:14:59.769000 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffe0de-9769-4e68-85a4-b687828d6b4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:59.770374 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffe0de-9769-4e68-85a4-b687828d6b4c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:59.772994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3dfc11-2692-45fc-a754-d8b67d1eb1cc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:59.779024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 41/84] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:14:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:14:59.779655 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffe0de-9769-4e68-85a4-b687828d6b4c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:59.809290 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0c6d3-1597-4802-8eb1-7a61718d8d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:59.809940 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0c6d3-1597-4802-8eb1-7a61718d8d41 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:59.810231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0c6d3-1597-4802-8eb1-7a61718d8d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:59.810512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0c6d3-1597-4802-8eb1-7a61718d8d41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:59.858432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:59.875304 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:14:59.880166 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0c6d3-1597-4802-8eb1-7a61718d8d41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:59.882321 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0c6d3-1597-4802-8eb1-7a61718d8d41 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:14:59.885370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffe0de-9769-4e68-85a4-b687828d6b4c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:14:59.926545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3b847-6f8a-4d1d-bbdc-8558905a50e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:14:59.927317 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3b847-6f8a-4d1d-bbdc-8558905a50e2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:14:59.927608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3b847-6f8a-4d1d-bbdc-8558905a50e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:14:59.927890 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3b847-6f8a-4d1d-bbdc-8558905a50e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:14:59.980833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 44/85] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:14:59.981748 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0c6d3-1597-4802-8eb1-7a61718d8d41 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:14:59.997559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3b847-6f8a-4d1d-bbdc-8558905a50e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:14:59.998980 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3b847-6f8a-4d1d-bbdc-8558905a50e2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:00.060443 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:00.078795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0c6d3-1597-4802-8eb1-7a61718d8d41 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:00.090508 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:00.098128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 42/86] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:00.098751 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3b847-6f8a-4d1d-bbdc-8558905a50e2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:00.114359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d5f1a-f179-4075-9034-564c5e51fcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:00.115022 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d5f1a-f179-4075-9034-564c5e51fcf1 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:00.115309 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d5f1a-f179-4075-9034-564c5e51fcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:00.115588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d5f1a-f179-4075-9034-564c5e51fcf1 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:00.116864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d5f1a-f179-4075-9034-564c5e51fcf1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:00.118299 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d5f1a-f179-4075-9034-564c5e51fcf1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:00.144522 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 45/87] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 18 10:14:59 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:15:00.145481 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d5f1a-f179-4075-9034-564c5e51fcf1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:00.207699 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3b847-6f8a-4d1d-bbdc-8558905a50e2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:00.247390 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e357c-7521-4809-9610-5795b6fca7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:00.248345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e357c-7521-4809-9610-5795b6fca7ea None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:00.248606 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e357c-7521-4809-9610-5795b6fca7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:00.248889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e357c-7521-4809-9610-5795b6fca7ea None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:00.250321 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e357c-7521-4809-9610-5795b6fca7ea None admin] RBAC: Authorizing `identity:get_project(project_id=09d730b36d8f4ab482dc9ed98757bb0b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:00.251824 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e357c-7521-4809-9610-5795b6fca7ea None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:00.253763 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 43/88] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:00 2022] GET /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:00.254456 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e357c-7521-4809-9610-5795b6fca7ea None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:00.292952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d5f1a-f179-4075-9034-564c5e51fcf1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:00.325489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:00.336615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef2138-dd84-4854-b27c-2dfa5cee83a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:00.338462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef2138-dd84-4854-b27c-2dfa5cee83a9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:00.338765 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef2138-dd84-4854-b27c-2dfa5cee83a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:00.339056 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef2138-dd84-4854-b27c-2dfa5cee83a9 None admin] PATH_INFO: `/v3/users/4eecd6829d5a4a319fa4af969eb3d39d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:00.340298 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef2138-dd84-4854-b27c-2dfa5cee83a9 None admin] RBAC: Authorizing `identity:get_user(user_id=4eecd6829d5a4a319fa4af969eb3d39d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:00.341831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef2138-dd84-4854-b27c-2dfa5cee83a9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:00.343922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 46/89] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:00 2022] GET /identity/v3/users/4eecd6829d5a4a319fa4af969eb3d39d => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:00.344973 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef2138-dd84-4854-b27c-2dfa5cee83a9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:00.355965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:00.371907 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e357c-7521-4809-9610-5795b6fca7ea None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:00.415739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ab6df-9aa6-4f92-9ebc-297eddf4bb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:00.416824 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ab6df-9aa6-4f92-9ebc-297eddf4bb26 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:00.417467 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ab6df-9aa6-4f92-9ebc-297eddf4bb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:00.417760 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ab6df-9aa6-4f92-9ebc-297eddf4bb26 None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:00.419257 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ab6df-9aa6-4f92-9ebc-297eddf4bb26 None admin] RBAC: Authorizing `identity:get_project(project_id=09d730b36d8f4ab482dc9ed98757bb0b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:00.420983 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ab6df-9aa6-4f92-9ebc-297eddf4bb26 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:00.423562 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 44/90] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:00 2022] GET /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:00.424419 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ab6df-9aa6-4f92-9ebc-297eddf4bb26 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:00.472034 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef2138-dd84-4854-b27c-2dfa5cee83a9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:00.513891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367484e-73c4-4465-b17b-919bc4f3643c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:00.514684 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367484e-73c4-4465-b17b-919bc4f3643c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:00.514976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367484e-73c4-4465-b17b-919bc4f3643c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:00.515246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367484e-73c4-4465-b17b-919bc4f3643c None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:00.516741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367484e-73c4-4465-b17b-919bc4f3643c None admin] RBAC: Authorizing `identity:get_project(project_id=09d730b36d8f4ab482dc9ed98757bb0b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:00.518387 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367484e-73c4-4465-b17b-919bc4f3643c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:00.520339 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 47/91] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:00 2022] GET /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:00.521277 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367484e-73c4-4465-b17b-919bc4f3643c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:00.544815 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ab6df-9aa6-4f92-9ebc-297eddf4bb26 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:00.579799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:00.586825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3215f6-ac52-4052-8286-6e682097e43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:00.587791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3215f6-ac52-4052-8286-6e682097e43d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:00.588255 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3215f6-ac52-4052-8286-6e682097e43d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:00.588742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3215f6-ac52-4052-8286-6e682097e43d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:00.589870 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3215f6-ac52-4052-8286-6e682097e43d None admin] RBAC: Adding query filter params (role.id=2820090979cb487db491f835e03e6471, scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b, user.id=4eecd6829d5a4a319fa4af969eb3d39d) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:00.590469 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3215f6-ac52-4052-8286-6e682097e43d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:00.592174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3215f6-ac52-4052-8286-6e682097e43d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:00.598945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 45/92] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:00 2022] GET /identity/v3/role_assignments?user.id=4eecd6829d5a4a319fa4af969eb3d39d&scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b&role.id=2820090979cb487db491f835e03e6471 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:00.599986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3215f6-ac52-4052-8286-6e682097e43d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:00.606017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:00.646114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367484e-73c4-4465-b17b-919bc4f3643c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:00.715705 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3215f6-ac52-4052-8286-6e682097e43d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:00.726558 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735b9f9-675d-4704-b0cb-7a797103fec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:00.727244 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735b9f9-675d-4704-b0cb-7a797103fec5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:00.727840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735b9f9-675d-4704-b0cb-7a797103fec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:00.728139 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735b9f9-675d-4704-b0cb-7a797103fec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:00.787191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66862382-9ebd-4970-b4fb-11317a282ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:00.787892 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66862382-9ebd-4970-b4fb-11317a282ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:00.788434 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66862382-9ebd-4970-b4fb-11317a282ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:00.788774 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66862382-9ebd-4970-b4fb-11317a282ed9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:00.789765 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66862382-9ebd-4970-b4fb-11317a282ed9 None admin] RBAC: Adding query filter params (role.id=8c039234db9640d0bdc0c2a629ff9e52, scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b, user.id=4eecd6829d5a4a319fa4af969eb3d39d) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:00.790091 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66862382-9ebd-4970-b4fb-11317a282ed9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:00.791837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66862382-9ebd-4970-b4fb-11317a282ed9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:00.804670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 46/93] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:00 2022] GET /identity/v3/role_assignments?user.id=4eecd6829d5a4a319fa4af969eb3d39d&scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b&role.id=8c039234db9640d0bdc0c2a629ff9e52 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:00.805558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66862382-9ebd-4970-b4fb-11317a282ed9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:00.815648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735b9f9-675d-4704-b0cb-7a797103fec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:00.817249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735b9f9-675d-4704-b0cb-7a797103fec5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:00.899834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:00.928748 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 48/94] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:00.933827 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:00.934313 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735b9f9-675d-4704-b0cb-7a797103fec5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:01.012920 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66862382-9ebd-4970-b4fb-11317a282ed9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:01.051633 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735b9f9-675d-4704-b0cb-7a797103fec5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:01.087008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5328eb-a120-4521-b5ee-9133a64eb721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:01.087698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5328eb-a120-4521-b5ee-9133a64eb721 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:01.087990 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5328eb-a120-4521-b5ee-9133a64eb721 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:01.088298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5328eb-a120-4521-b5ee-9133a64eb721 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:01.089426 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5328eb-a120-4521-b5ee-9133a64eb721 None admin] RBAC: Adding query filter params (role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b, user.id=06ec5a62dde8447fa4c9a00a0f1828f8) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:01.089757 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5328eb-a120-4521-b5ee-9133a64eb721 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:01.090068 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7eafb-ef2e-420c-92c8-54fc37382509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:01.091014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7eafb-ef2e-420c-92c8-54fc37382509 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:01.091231 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5328eb-a120-4521-b5ee-9133a64eb721 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:01.091431 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7eafb-ef2e-420c-92c8-54fc37382509 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:01.091787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7eafb-ef2e-420c-92c8-54fc37382509 None admin] PATH_INFO: `/v3/projects/bcfc47788c5742db9a4a501fd70663de` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:01.093107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7eafb-ef2e-420c-92c8-54fc37382509 None admin] RBAC: Authorizing `identity:get_project(project_id=bcfc47788c5742db9a4a501fd70663de)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:01.094659 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7eafb-ef2e-420c-92c8-54fc37382509 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:01.096965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 49/95] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:00 2022] GET /identity/v3/projects/bcfc47788c5742db9a4a501fd70663de => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:01.104104 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 47/96] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:00 2022] GET /identity/v3/role_assignments?user.id=06ec5a62dde8447fa4c9a00a0f1828f8&scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:01.105369 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7eafb-ef2e-420c-92c8-54fc37382509 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:01.166103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:01.252776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7eafb-ef2e-420c-92c8-54fc37382509 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:01.295548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ebcb5-9049-463b-9a75-e804b81fda1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:01.296230 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ebcb5-9049-463b-9a75-e804b81fda1e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:01.296534 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ebcb5-9049-463b-9a75-e804b81fda1e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:01.296828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ebcb5-9049-463b-9a75-e804b81fda1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:01.297693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ebcb5-9049-463b-9a75-e804b81fda1e None admin] RBAC: Adding query filter params (role.id=2820090979cb487db491f835e03e6471, scope.project.id=bcfc47788c5742db9a4a501fd70663de, user.id=4eecd6829d5a4a319fa4af969eb3d39d) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:01.298012 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ebcb5-9049-463b-9a75-e804b81fda1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:01.299559 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ebcb5-9049-463b-9a75-e804b81fda1e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:01.306369 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 50/97] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:01 2022] GET /identity/v3/role_assignments?user.id=4eecd6829d5a4a319fa4af969eb3d39d&scope.project.id=bcfc47788c5742db9a4a501fd70663de&role.id=2820090979cb487db491f835e03e6471 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:02.579999 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b086ea6-4ecb-48f4-b5b7-d309fad2c64b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:02.580709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b086ea6-4ecb-48f4-b5b7-d309fad2c64b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:02.580709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b086ea6-4ecb-48f4-b5b7-d309fad2c64b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:02.581898 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 48/98] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:02.587935 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f255b-992a-4a15-a1bb-c9f73a79e2f7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:02.588430 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f255b-992a-4a15-a1bb-c9f73a79e2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:02.588617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f255b-992a-4a15-a1bb-c9f73a79e2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:02.609615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5e5f255b-992a-4a15-a1bb-c9f73a79e2f7 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:02.610843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5e5f255b-992a-4a15-a1bb-c9f73a79e2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:02.695733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 51/99] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:02.735297 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b086ea6-4ecb-48f4-b5b7-d309fad2c64b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:02.853255 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b086ea6-4ecb-48f4-b5b7-d309fad2c64b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:02.886901 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:02.902595 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad251b1-3f51-45dc-95ea-54b696cb79ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:02.903405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad251b1-3f51-45dc-95ea-54b696cb79ce None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:02.903703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad251b1-3f51-45dc-95ea-54b696cb79ce None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:02.903991 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad251b1-3f51-45dc-95ea-54b696cb79ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:02.918322 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad251b1-3f51-45dc-95ea-54b696cb79ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:02.926289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-3ad251b1-3f51-45dc-95ea-54b696cb79ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:15:02.926289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:02.926289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:02.926289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:02.926289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:02.926289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:02.926289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:02.926289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:02.926289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:02.926289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:02.926289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:02.926289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:02.926289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:02.926289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:02.926289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:02.926289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:02.928266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:02.928266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:02.928266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:02.928266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:02.928266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:02.928266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:02.928266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:02.928266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:02.928266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:02.928266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:02.928266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:02.928266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:02.928266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:02.928266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:02.928266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:02.928266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:02.930803 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:02.930803 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:02.930803 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:02.930803 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:02.930803 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:02.930803 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:02.930803 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:02.930803 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:02.930803 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:02.930803 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:02.930803 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:15:02.930803 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:02.932206 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 49/100] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:02.938257 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f255b-992a-4a15-a1bb-c9f73a79e2f7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:02.950339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed145f63-da89-403f-b06d-d1fdf2ceebcd None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:02.950637 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed145f63-da89-403f-b06d-d1fdf2ceebcd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:02.950933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed145f63-da89-403f-b06d-d1fdf2ceebcd None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:02.952321 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 50/101] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:02.957625 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab27d0a-66a1-4aa0-8d66-0a350dcf7fb8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:02.957923 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab27d0a-66a1-4aa0-8d66-0a350dcf7fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:02.958211 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab27d0a-66a1-4aa0-8d66-0a350dcf7fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:02.996675 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fab27d0a-66a1-4aa0-8d66-0a350dcf7fb8 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:02.996971 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:02.997985 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fab27d0a-66a1-4aa0-8d66-0a350dcf7fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:03.099608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 51/102] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:03.133135 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab27d0a-66a1-4aa0-8d66-0a350dcf7fb8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:03.140259 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f255b-992a-4a15-a1bb-c9f73a79e2f7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:03.230301 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b5ba4-ed12-4ca6-b8b2-49cd75d0f154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:03.230996 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b5ba4-ed12-4ca6-b8b2-49cd75d0f154 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:03.231290 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b5ba4-ed12-4ca6-b8b2-49cd75d0f154 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:03.231577 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b5ba4-ed12-4ca6-b8b2-49cd75d0f154 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:03.232353 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b5ba4-ed12-4ca6-b8b2-49cd75d0f154 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:03.232699 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b5ba4-ed12-4ca6-b8b2-49cd75d0f154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:03.234118 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b5ba4-ed12-4ca6-b8b2-49cd75d0f154 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:03.251509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 52/103] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:03.257433 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b5ba4-ed12-4ca6-b8b2-49cd75d0f154 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:03.374652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab27d0a-66a1-4aa0-8d66-0a350dcf7fb8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:03.408205 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:03.425796 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:03.435798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b5ba4-ed12-4ca6-b8b2-49cd75d0f154 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:03.469225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038d11a-0ee6-432b-aa49-38ded83e7172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:03.470046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038d11a-0ee6-432b-aa49-38ded83e7172 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:03.470335 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038d11a-0ee6-432b-aa49-38ded83e7172 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:03.470615 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038d11a-0ee6-432b-aa49-38ded83e7172 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:03.471841 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038d11a-0ee6-432b-aa49-38ded83e7172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:03.473357 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038d11a-0ee6-432b-aa49-38ded83e7172 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:03.475019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 52/104] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:03.485465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038d11a-0ee6-432b-aa49-38ded83e7172 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:03.505840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5234d2-fb2c-408a-8a5f-8b2a5ca06496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:03.507278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5234d2-fb2c-408a-8a5f-8b2a5ca06496 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:03.507567 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5234d2-fb2c-408a-8a5f-8b2a5ca06496 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:03.507848 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5234d2-fb2c-408a-8a5f-8b2a5ca06496 None admin] PATH_INFO: `/v3/users/cc013382f7504c65a526dd7f68c9e78f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:03.537782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5234d2-fb2c-408a-8a5f-8b2a5ca06496 None admin] RBAC: Authorizing `identity:get_user(user_id=cc013382f7504c65a526dd7f68c9e78f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:03.540096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5234d2-fb2c-408a-8a5f-8b2a5ca06496 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:03.542228 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 53/105] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:03 2022] GET /identity/v3/users/cc013382f7504c65a526dd7f68c9e78f => generated 289 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:03.547440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5234d2-fb2c-408a-8a5f-8b2a5ca06496 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:03.695574 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5234d2-fb2c-408a-8a5f-8b2a5ca06496 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:03.747916 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb96ed-71f6-4b7e-a4b5-79b560e61b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:03.749802 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb96ed-71f6-4b7e-a4b5-79b560e61b6d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:03.751335 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb96ed-71f6-4b7e-a4b5-79b560e61b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:03.751818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb96ed-71f6-4b7e-a4b5-79b560e61b6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:03.754766 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038d11a-0ee6-432b-aa49-38ded83e7172 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:03.788216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:03.838354 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb96ed-71f6-4b7e-a4b5-79b560e61b6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:03.840418 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb96ed-71f6-4b7e-a4b5-79b560e61b6d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:03.865819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd746c6-8371-4190-aa78-b848751fff30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:03.866559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd746c6-8371-4190-aa78-b848751fff30 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:03.867019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd746c6-8371-4190-aa78-b848751fff30 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:03.867474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd746c6-8371-4190-aa78-b848751fff30 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:03.868340 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd746c6-8371-4190-aa78-b848751fff30 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:03.869980 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd746c6-8371-4190-aa78-b848751fff30 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:03.953872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 54/106] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 10:15:03.954462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 53/107] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 18 10:15:03 2022] POST /identity/v3/projects => generated 302 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:15:03.955790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bed1e-e167-495c-b6ad-a9e3e376cf7d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:03.961203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bed1e-e167-495c-b6ad-a9e3e376cf7d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:03.961803 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bed1e-e167-495c-b6ad-a9e3e376cf7d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:03.963453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 55/108] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:03.964639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd746c6-8371-4190-aa78-b848751fff30 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:03.974506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271f125-0b41-4b5b-aad6-374b6a9ddc5b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:03.975143 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271f125-0b41-4b5b-aad6-374b6a9ddc5b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:03.976186 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271f125-0b41-4b5b-aad6-374b6a9ddc5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:04.006495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5271f125-0b41-4b5b-aad6-374b6a9ddc5b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:04.008388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5271f125-0b41-4b5b-aad6-374b6a9ddc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:04.071340 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:04.116915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:04.181489 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 56/109] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:04.184527 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d297c-d81f-4b0d-81f2-9374abf02e55 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:04.184527 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d297c-d81f-4b0d-81f2-9374abf02e55 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:04.184527 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d297c-d81f-4b0d-81f2-9374abf02e55 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:04.185003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 57/110] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:04.199303 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ba23e-582d-4e5e-84b9-39421ad3da2c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:04.199825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ba23e-582d-4e5e-84b9-39421ad3da2c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:04.200329 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ba23e-582d-4e5e-84b9-39421ad3da2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:04.233980 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e60ba23e-582d-4e5e-84b9-39421ad3da2c None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:04.235557 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e60ba23e-582d-4e5e-84b9-39421ad3da2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:04.245758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd746c6-8371-4190-aa78-b848751fff30 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:04.289503 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea26fc-bc6e-4e13-9918-ea277d9b3186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:04.290717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea26fc-bc6e-4e13-9918-ea277d9b3186 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:04.291016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea26fc-bc6e-4e13-9918-ea277d9b3186 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:04.291305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea26fc-bc6e-4e13-9918-ea277d9b3186 None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:04.292830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea26fc-bc6e-4e13-9918-ea277d9b3186 None admin] RBAC: Authorizing `identity:get_project(project_id=09d730b36d8f4ab482dc9ed98757bb0b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:04.294852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea26fc-bc6e-4e13-9918-ea277d9b3186 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:04.297132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 54/111] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:03 2022] GET /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b => generated 298 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:04.299661 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea26fc-bc6e-4e13-9918-ea277d9b3186 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:04.320649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 58/112] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:04.321918 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ba23e-582d-4e5e-84b9-39421ad3da2c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:04.434697 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea26fc-bc6e-4e13-9918-ea277d9b3186 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:04.437462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:04.438857 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ba23e-582d-4e5e-84b9-39421ad3da2c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:04.459141 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:04.477122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c48c34c-1521-40bd-a4e5-f0d17900fd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:04.478186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48c34c-1521-40bd-a4e5-f0d17900fd79 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:04.478473 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48c34c-1521-40bd-a4e5-f0d17900fd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:04.478749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48c34c-1521-40bd-a4e5-f0d17900fd79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:04.480056 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004f8a1-933d-4805-9cb0-288cfd46c456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:04.481365 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004f8a1-933d-4805-9cb0-288cfd46c456 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:04.481849 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004f8a1-933d-4805-9cb0-288cfd46c456 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:04.482317 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004f8a1-933d-4805-9cb0-288cfd46c456 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:04.483350 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004f8a1-933d-4805-9cb0-288cfd46c456 None admin] RBAC: Adding query filter params (role.id=e6c6a363f7034bd0a72b72ffe324aab8, scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b, user.id=cc013382f7504c65a526dd7f68c9e78f) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:04.483879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004f8a1-933d-4805-9cb0-288cfd46c456 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:04.485623 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004f8a1-933d-4805-9cb0-288cfd46c456 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:04.489709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c48c34c-1521-40bd-a4e5-f0d17900fd79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:04.492773 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 59/113] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:04 2022] GET /identity/v3/role_assignments?user.id=cc013382f7504c65a526dd7f68c9e78f&scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b&role.id=e6c6a363f7034bd0a72b72ffe324aab8 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:04.497359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004f8a1-933d-4805-9cb0-288cfd46c456 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:04.498248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8c48c34c-1521-40bd-a4e5-f0d17900fd79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:04.498248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:04.498248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:04.498248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:04.498248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:04.498248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:04.498248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:04.498248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:04.498248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:04.498248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:04.498248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:04.498248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:04.498248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:04.498248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:04.498248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:04.498248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:04.499063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:04.499063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:04.499063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:04.499063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:04.499063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:04.499063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:04.499063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:04.499063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:04.499063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:04.499063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:04.499063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:04.499063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:04.499063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:04.499063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:04.499063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:04.499063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:04.499843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:04.499843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:04.499843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:04.499843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:04.499843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:04.499843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:04.499843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:04.499843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:04.499843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:04.499843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:04.499843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:04.499843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:04.499843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 55/114] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:04.506083 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b5e99-b72a-4cc9-ab92-f93bf58ec89c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:04.506401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b5e99-b72a-4cc9-ab92-f93bf58ec89c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:04.507236 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b5e99-b72a-4cc9-ab92-f93bf58ec89c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:04.508766 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 56/115] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:04.510458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c2e60-5097-46ff-8dbb-9e8f362571f1 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:04.510757 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c2e60-5097-46ff-8dbb-9e8f362571f1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:04.511042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c2e60-5097-46ff-8dbb-9e8f362571f1 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:04.513128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 57/116] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:04.513578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c2e60-5097-46ff-8dbb-9e8f362571f1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:04.635392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004f8a1-933d-4805-9cb0-288cfd46c456 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:04.644849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c2e60-5097-46ff-8dbb-9e8f362571f1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:04.680250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ce681-ba82-4a0f-b9f8-9ac85b97856f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:04.681365 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ce681-ba82-4a0f-b9f8-9ac85b97856f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:04.681656 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ce681-ba82-4a0f-b9f8-9ac85b97856f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:04.681939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ce681-ba82-4a0f-b9f8-9ac85b97856f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:04.688349 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03274a2e-3add-46b8-a012-33115105fee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:04.689087 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03274a2e-3add-46b8-a012-33115105fee3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:04.689436 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03274a2e-3add-46b8-a012-33115105fee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:04.689727 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03274a2e-3add-46b8-a012-33115105fee3 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:04.690498 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03274a2e-3add-46b8-a012-33115105fee3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:04.690874 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03274a2e-3add-46b8-a012-33115105fee3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:04.692325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03274a2e-3add-46b8-a012-33115105fee3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:04.694523 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ce681-ba82-4a0f-b9f8-9ac85b97856f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:04.701957 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 58/117] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:04.703805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-670ce681-ba82-4a0f-b9f8-9ac85b97856f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:15:04.703805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:04.703805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:04.703805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:04.703805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:04.703805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:04.703805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:04.703805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:04.703805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:04.703805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:04.703805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:04.703805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:04.703805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:04.703805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:04.703805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:04.703805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:04.705026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:04.705026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:04.705026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:04.705026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:04.705026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:04.705026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:04.705026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:04.705026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:04.705026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:04.705026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:04.705026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:04.705026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:04.705026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:04.705026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:04.705026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:04.705026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:04.705793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:04.705793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:04.705793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:04.705793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:04.705793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:04.705793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:04.705793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:04.705793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:04.705793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:04.705793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:04.705793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:15:04.705793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:04.705793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 60/118] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:15:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:04.706912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28279ce-43f4-4fe8-9a19-8cb72ff2f8f6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:04.706912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28279ce-43f4-4fe8-9a19-8cb72ff2f8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:04.707192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28279ce-43f4-4fe8-9a19-8cb72ff2f8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:04.712528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc0b8c-8695-438c-adf8-5eb6aaad1369 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:04.712528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc0b8c-8695-438c-adf8-5eb6aaad1369 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:04.712528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc0b8c-8695-438c-adf8-5eb6aaad1369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:04.732823 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c28279ce-43f4-4fe8-9a19-8cb72ff2f8f6 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:04.733869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-12fc0b8c-8695-438c-adf8-5eb6aaad1369 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:04.734126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c28279ce-43f4-4fe8-9a19-8cb72ff2f8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:04.735060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-12fc0b8c-8695-438c-adf8-5eb6aaad1369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:04.755896 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:04.782697 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:04.823209 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 61/119] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:04.824163 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28279ce-43f4-4fe8-9a19-8cb72ff2f8f6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:04.828584 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 59/120] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:04.829537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc0b8c-8695-438c-adf8-5eb6aaad1369 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:04.946712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28279ce-43f4-4fe8-9a19-8cb72ff2f8f6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:04.975203 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc0b8c-8695-438c-adf8-5eb6aaad1369 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:04.985586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91319277-696f-4444-acc5-907e205bd5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:04.986527 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91319277-696f-4444-acc5-907e205bd5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:04.986822 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91319277-696f-4444-acc5-907e205bd5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:04.987107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91319277-696f-4444-acc5-907e205bd5a8 None admin] PATH_INFO: `/v3/users/4eecd6829d5a4a319fa4af969eb3d39d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:04.988393 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91319277-696f-4444-acc5-907e205bd5a8 None admin] RBAC: Authorizing `identity:get_user(user_id=4eecd6829d5a4a319fa4af969eb3d39d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:04.989935 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91319277-696f-4444-acc5-907e205bd5a8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:04.996244 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 62/121] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:04 2022] GET /identity/v3/users/4eecd6829d5a4a319fa4af969eb3d39d => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:04.997162 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91319277-696f-4444-acc5-907e205bd5a8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:05.019880 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005e096-c626-4f8c-9b07-d67567729c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:05.021063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005e096-c626-4f8c-9b07-d67567729c22 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:05.021362 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005e096-c626-4f8c-9b07-d67567729c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:05.021650 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005e096-c626-4f8c-9b07-d67567729c22 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:05.023033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005e096-c626-4f8c-9b07-d67567729c22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:05.023352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005e096-c626-4f8c-9b07-d67567729c22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:05.024958 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005e096-c626-4f8c-9b07-d67567729c22 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:05.033979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 60/122] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:05.034932 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005e096-c626-4f8c-9b07-d67567729c22 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:05.045815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:05.056282 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:05.123787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91319277-696f-4444-acc5-907e205bd5a8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:05.164529 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005e096-c626-4f8c-9b07-d67567729c22 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:05.176724 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde0dc1-4072-4799-a005-d85e947d9be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:05.180869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde0dc1-4072-4799-a005-d85e947d9be8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:05.180869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde0dc1-4072-4799-a005-d85e947d9be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:05.180869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde0dc1-4072-4799-a005-d85e947d9be8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:05.200469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde0dc1-4072-4799-a005-d85e947d9be8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:05.208827 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f3466-c8b6-40c7-a42a-5e63f7bac88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:05.209718 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f3466-c8b6-40c7-a42a-5e63f7bac88d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:05.210600 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f3466-c8b6-40c7-a42a-5e63f7bac88d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:05.210940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f3466-c8b6-40c7-a42a-5e63f7bac88d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:05.212891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-efde0dc1-4072-4799-a005-d85e947d9be8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:05.212891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:05.212891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:05.212891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:05.212891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:05.212891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:05.212891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:05.212891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:05.212891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:05.212891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:05.212891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:05.212891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:05.212891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:05.212891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:05.212891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:05.212891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:05.213787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:05.213787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:05.213787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:05.213787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:05.213787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:05.213787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:05.213787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:05.213787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:05.213787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:05.213787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:05.213787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:05.213787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:05.213787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:05.213787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:05.213787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:05.213787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:05.214547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:05.214547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:05.214547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:05.214547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:05.214547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:05.214547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:05.214547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:05.214547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:05.214547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:05.214547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:05.214547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:05.214547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:05.214547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 63/123] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:05.215172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde0dc1-4072-4799-a005-d85e947d9be8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:05.223656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f3466-c8b6-40c7-a42a-5e63f7bac88d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:05.233144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ad4f3466-c8b6-40c7-a42a-5e63f7bac88d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:05.233144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:05.233144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:05.233144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:05.233144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:05.233144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:05.233144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:05.233144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:05.233144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:05.233144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:05.233144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:05.233144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:05.233144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:05.233144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:05.233144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:05.233144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:05.233987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:05.233987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:05.233987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:05.233987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:05.233987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:05.233987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:05.233987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:05.233987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:05.233987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:05.233987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:05.233987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:05.233987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:05.233987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:05.233987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:05.233987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:05.233987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:05.234756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:05.234756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:05.234756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:05.234756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:05.234756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:05.234756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:05.234756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:05.234756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:05.234756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:05.234756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:05.234756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:05.234756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:05.234756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 61/124] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:15:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:05.240123 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f3466-c8b6-40c7-a42a-5e63f7bac88d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:05.315372 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:05.319097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde0dc1-4072-4799-a005-d85e947d9be8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:05.330846 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:05.354891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae605d6-2c69-4784-ad2d-86bf84b05389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:05.355858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae605d6-2c69-4784-ad2d-86bf84b05389 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:05.356215 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae605d6-2c69-4784-ad2d-86bf84b05389 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:05.356598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae605d6-2c69-4784-ad2d-86bf84b05389 None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:05.357868 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae605d6-2c69-4784-ad2d-86bf84b05389 None admin] RBAC: Authorizing `identity:get_project(project_id=09d730b36d8f4ab482dc9ed98757bb0b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:05.358067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f3466-c8b6-40c7-a42a-5e63f7bac88d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:05.359494 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae605d6-2c69-4784-ad2d-86bf84b05389 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:05.361489 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 64/125] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:05 2022] GET /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:05.362169 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae605d6-2c69-4784-ad2d-86bf84b05389 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:05.405670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a087b4-105a-4c6e-9ec0-b4ecc5dd97ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:05.406870 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a087b4-105a-4c6e-9ec0-b4ecc5dd97ec None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:05.407325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a087b4-105a-4c6e-9ec0-b4ecc5dd97ec None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:05.407616 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a087b4-105a-4c6e-9ec0-b4ecc5dd97ec None admin] PATH_INFO: `/v3/users/06ec5a62dde8447fa4c9a00a0f1828f8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:05.409429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a087b4-105a-4c6e-9ec0-b4ecc5dd97ec None admin] RBAC: Authorizing `identity:get_user(user_id=06ec5a62dde8447fa4c9a00a0f1828f8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:05.411137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a087b4-105a-4c6e-9ec0-b4ecc5dd97ec None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:05.414265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 62/126] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:05 2022] GET /identity/v3/users/06ec5a62dde8447fa4c9a00a0f1828f8 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:05.415054 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a087b4-105a-4c6e-9ec0-b4ecc5dd97ec None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:05.488760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae605d6-2c69-4784-ad2d-86bf84b05389 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:05.525837 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f4b19-7b6d-4177-9102-39034d28ebd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:05.526530 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f4b19-7b6d-4177-9102-39034d28ebd3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:05.526825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f4b19-7b6d-4177-9102-39034d28ebd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:05.527112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f4b19-7b6d-4177-9102-39034d28ebd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:05.527880 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f4b19-7b6d-4177-9102-39034d28ebd3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:05.528253 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f4b19-7b6d-4177-9102-39034d28ebd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:05.529752 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f4b19-7b6d-4177-9102-39034d28ebd3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:05.539538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 65/127] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:05.540571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f4b19-7b6d-4177-9102-39034d28ebd3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:05.560199 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a087b4-105a-4c6e-9ec0-b4ecc5dd97ec None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:05.570973 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:05.605074 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0911380-f002-42d0-9004-7a3798ff1e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:05.605802 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0911380-f002-42d0-9004-7a3798ff1e97 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:05.606099 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0911380-f002-42d0-9004-7a3798ff1e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:05.606383 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0911380-f002-42d0-9004-7a3798ff1e97 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:05.607268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0911380-f002-42d0-9004-7a3798ff1e97 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:05.607602 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0911380-f002-42d0-9004-7a3798ff1e97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:05.609110 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0911380-f002-42d0-9004-7a3798ff1e97 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:05.621067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 63/128] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:15:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:05.621896 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0911380-f002-42d0-9004-7a3798ff1e97 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:05.631503 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:05.659356 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f4b19-7b6d-4177-9102-39034d28ebd3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:05.696156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338b03f-5f88-4405-9970-2fd5ccc03e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:05.697491 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338b03f-5f88-4405-9970-2fd5ccc03e1d None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:05.697974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338b03f-5f88-4405-9970-2fd5ccc03e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:05.698464 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338b03f-5f88-4405-9970-2fd5ccc03e1d None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b/users/4eecd6829d5a4a319fa4af969eb3d39d/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:05.700859 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338b03f-5f88-4405-9970-2fd5ccc03e1d None admin] RBAC: Authorizing `identity:create_grant(project_id=09d730b36d8f4ab482dc9ed98757bb0b, user_id=4eecd6829d5a4a319fa4af969eb3d39d, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:05.702551 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338b03f-5f88-4405-9970-2fd5ccc03e1d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:05.723180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 66/129] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:15:05 2022] PUT /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b/users/4eecd6829d5a4a319fa4af969eb3d39d/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:15:05.725277 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338b03f-5f88-4405-9970-2fd5ccc03e1d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:05.744597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0911380-f002-42d0-9004-7a3798ff1e97 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:05.788968 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9364e-708a-4932-94a9-9d75d04ca8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:05.789987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9364e-708a-4932-94a9-9d75d04ca8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:05.790181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9364e-708a-4932-94a9-9d75d04ca8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:05.790463 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9364e-708a-4932-94a9-9d75d04ca8b0 None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:05.791733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9364e-708a-4932-94a9-9d75d04ca8b0 None admin] RBAC: Authorizing `identity:get_project(project_id=09d730b36d8f4ab482dc9ed98757bb0b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:05.793610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9364e-708a-4932-94a9-9d75d04ca8b0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:05.795874 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 64/130] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:05 2022] GET /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:05.797053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9364e-708a-4932-94a9-9d75d04ca8b0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:05.841191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:05.857857 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338b03f-5f88-4405-9970-2fd5ccc03e1d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:05.884586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:05.897962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5583c-11f3-4985-bd02-bf18b612419f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:05.898921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5583c-11f3-4985-bd02-bf18b612419f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:05.899217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5583c-11f3-4985-bd02-bf18b612419f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:05.899504 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5583c-11f3-4985-bd02-bf18b612419f None admin] PATH_INFO: `/v3/users/4eecd6829d5a4a319fa4af969eb3d39d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:05.900835 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5583c-11f3-4985-bd02-bf18b612419f None admin] RBAC: Authorizing `identity:get_user(user_id=4eecd6829d5a4a319fa4af969eb3d39d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:05.902389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5583c-11f3-4985-bd02-bf18b612419f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:05.904264 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 67/131] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:05 2022] GET /identity/v3/users/4eecd6829d5a4a319fa4af969eb3d39d => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:05.905072 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5583c-11f3-4985-bd02-bf18b612419f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:05.916328 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9364e-708a-4932-94a9-9d75d04ca8b0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:05.954472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115f414-b67b-4a63-b957-0af2e30135bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:05.955769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115f414-b67b-4a63-b957-0af2e30135bc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:05.956267 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115f414-b67b-4a63-b957-0af2e30135bc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:05.956795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115f414-b67b-4a63-b957-0af2e30135bc None admin] PATH_INFO: `/v3/users/4eecd6829d5a4a319fa4af969eb3d39d` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:05.958353 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1115f414-b67b-4a63-b957-0af2e30135bc None admin] RBAC: Authorizing `identity:get_user(user_id=4eecd6829d5a4a319fa4af969eb3d39d)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:05.960109 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1115f414-b67b-4a63-b957-0af2e30135bc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:05.962093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 65/132] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:05 2022] GET /identity/v3/users/4eecd6829d5a4a319fa4af969eb3d39d => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:05.963074 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115f414-b67b-4a63-b957-0af2e30135bc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:06.024531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5583c-11f3-4985-bd02-bf18b612419f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:06.058860 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3765b9-0fba-4e93-a348-627b1130f1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:06.059767 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3765b9-0fba-4e93-a348-627b1130f1fd None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:06.060059 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3765b9-0fba-4e93-a348-627b1130f1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:06.060342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3765b9-0fba-4e93-a348-627b1130f1fd None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b/users/06ec5a62dde8447fa4c9a00a0f1828f8/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:06.062378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3765b9-0fba-4e93-a348-627b1130f1fd None admin] RBAC: Authorizing `identity:create_grant(project_id=09d730b36d8f4ab482dc9ed98757bb0b, user_id=06ec5a62dde8447fa4c9a00a0f1828f8, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:06.063910 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3765b9-0fba-4e93-a348-627b1130f1fd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:06.081308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 68/133] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:15:05 2022] PUT /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b/users/06ec5a62dde8447fa4c9a00a0f1828f8/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:15:06.081995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3765b9-0fba-4e93-a348-627b1130f1fd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:06.104932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:06.120125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115f414-b67b-4a63-b957-0af2e30135bc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:06.167230 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf1762-1cc3-4fb7-9159-ed03224b3108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:06.168169 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf1762-1cc3-4fb7-9159-ed03224b3108 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:06.168455 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf1762-1cc3-4fb7-9159-ed03224b3108 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:06.168777 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf1762-1cc3-4fb7-9159-ed03224b3108 None admin] PATH_INFO: `/v3/projects/bcfc47788c5742db9a4a501fd70663de` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:06.170153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf1762-1cc3-4fb7-9159-ed03224b3108 None admin] RBAC: Authorizing `identity:get_project(project_id=bcfc47788c5742db9a4a501fd70663de)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:06.171807 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf1762-1cc3-4fb7-9159-ed03224b3108 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:06.173875 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 66/134] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:05 2022] GET /identity/v3/projects/bcfc47788c5742db9a4a501fd70663de => generated 312 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:06.192550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf1762-1cc3-4fb7-9159-ed03224b3108 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:06.202719 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:06.252808 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3765b9-0fba-4e93-a348-627b1130f1fd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:06.309494 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf1762-1cc3-4fb7-9159-ed03224b3108 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:06.321837 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ad2ac-ffaa-4e99-b2b5-9ad855631c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:06.322736 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ad2ac-ffaa-4e99-b2b5-9ad855631c44 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:06.323030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ad2ac-ffaa-4e99-b2b5-9ad855631c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:06.323312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ad2ac-ffaa-4e99-b2b5-9ad855631c44 None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:06.324614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ad2ac-ffaa-4e99-b2b5-9ad855631c44 None admin] RBAC: Authorizing `identity:get_project(project_id=09d730b36d8f4ab482dc9ed98757bb0b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:06.326228 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ad2ac-ffaa-4e99-b2b5-9ad855631c44 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:06.328226 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 69/135] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:06 2022] GET /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:06.333983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ad2ac-ffaa-4e99-b2b5-9ad855631c44 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:06.347932 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7eb87-54c8-4a76-aadf-717d26fb57ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:06.348878 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7eb87-54c8-4a76-aadf-717d26fb57ba None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:06.349166 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7eb87-54c8-4a76-aadf-717d26fb57ba None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:06.349449 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7eb87-54c8-4a76-aadf-717d26fb57ba None admin] PATH_INFO: `/v3/projects/bcfc47788c5742db9a4a501fd70663de/users/4eecd6829d5a4a319fa4af969eb3d39d/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:06.351571 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7eb87-54c8-4a76-aadf-717d26fb57ba None admin] RBAC: Authorizing `identity:create_grant(project_id=bcfc47788c5742db9a4a501fd70663de, user_id=4eecd6829d5a4a319fa4af969eb3d39d, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:06.353360 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7eb87-54c8-4a76-aadf-717d26fb57ba None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:06.395529 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 67/136] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:15:06 2022] PUT /identity/v3/projects/bcfc47788c5742db9a4a501fd70663de/users/4eecd6829d5a4a319fa4af969eb3d39d/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:15:06.453517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:06.473600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ad2ac-ffaa-4e99-b2b5-9ad855631c44 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:06.513594 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab06bc-5c22-43bb-9167-66309864dec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:06.514605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab06bc-5c22-43bb-9167-66309864dec9 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:06.514952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab06bc-5c22-43bb-9167-66309864dec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:06.515244 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab06bc-5c22-43bb-9167-66309864dec9 None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b/users/4eecd6829d5a4a319fa4af969eb3d39d/roles/8c039234db9640d0bdc0c2a629ff9e52` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:06.517600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab06bc-5c22-43bb-9167-66309864dec9 None admin] RBAC: Authorizing `identity:create_grant(project_id=09d730b36d8f4ab482dc9ed98757bb0b, user_id=4eecd6829d5a4a319fa4af969eb3d39d, role_id=8c039234db9640d0bdc0c2a629ff9e52)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:06.519114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab06bc-5c22-43bb-9167-66309864dec9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:06.551580 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 70/137] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:15:06 2022] PUT /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b/users/4eecd6829d5a4a319fa4af969eb3d39d/roles/8c039234db9640d0bdc0c2a629ff9e52 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:15:06.598300 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbad479-d097-4c18-b8ab-98519c9b2e18 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:06.598617 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbad479-d097-4c18-b8ab-98519c9b2e18 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:06.598902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbad479-d097-4c18-b8ab-98519c9b2e18 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:06.600424 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 68/138] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:06.607067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941970c0-c8f5-4385-b423-b5f6157132aa None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:06.607750 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941970c0-c8f5-4385-b423-b5f6157132aa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:06.608318 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941970c0-c8f5-4385-b423-b5f6157132aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:06.627436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-941970c0-c8f5-4385-b423-b5f6157132aa None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:06.629052 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-941970c0-c8f5-4385-b423-b5f6157132aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:06.714781 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 71/139] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:06.761327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbad479-d097-4c18-b8ab-98519c9b2e18 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:06.848804 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:06.892957 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbad479-d097-4c18-b8ab-98519c9b2e18 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:06.939416 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c696a1-c849-4b89-b871-b53fc3a611a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:06.940219 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c696a1-c849-4b89-b871-b53fc3a611a8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:06.940524 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c696a1-c849-4b89-b871-b53fc3a611a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:06.940805 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c696a1-c849-4b89-b871-b53fc3a611a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:06.945571 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c696a1-c849-4b89-b871-b53fc3a611a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:06.947045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c696a1-c849-4b89-b871-b53fc3a611a8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:06.948734 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 69/140] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:06.954419 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941970c0-c8f5-4385-b423-b5f6157132aa None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:06.998452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968fec2-b919-4cc7-be4a-73e770209b4a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:06.998744 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968fec2-b919-4cc7-be4a-73e770209b4a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:06.998935 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968fec2-b919-4cc7-be4a-73e770209b4a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:07.000391 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 70/141] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:15:07.011061 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b460e-8933-4064-97dc-8247188fac6a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:07.011357 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b460e-8933-4064-97dc-8247188fac6a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:07.011661 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b460e-8933-4064-97dc-8247188fac6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:07.039075 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f32b460e-8933-4064-97dc-8247188fac6a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:07.040292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f32b460e-8933-4064-97dc-8247188fac6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:07.081885 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:07.122822 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941970c0-c8f5-4385-b423-b5f6157132aa None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:07.177262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ceadae-53fb-48a8-9198-4b77fa12767a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:07.179233 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ceadae-53fb-48a8-9198-4b77fa12767a None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:07.180380 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ceadae-53fb-48a8-9198-4b77fa12767a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:07.181041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ceadae-53fb-48a8-9198-4b77fa12767a None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:07.183094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ceadae-53fb-48a8-9198-4b77fa12767a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:07.186170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ceadae-53fb-48a8-9198-4b77fa12767a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:07.199558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 71/142] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:07.223602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 72/143] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 18 10:15:06 2022] POST /identity/v3/users => generated 283 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:15:07.242542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b460e-8933-4064-97dc-8247188fac6a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:07.340933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:07.383256 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b460e-8933-4064-97dc-8247188fac6a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:07.427621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d9d4e-e1a8-4ef8-980b-38c8a9961ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:07.428427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d9d4e-e1a8-4ef8-980b-38c8a9961ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:07.428784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d9d4e-e1a8-4ef8-980b-38c8a9961ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:07.429064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d9d4e-e1a8-4ef8-980b-38c8a9961ed7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:07.440534 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d9d4e-e1a8-4ef8-980b-38c8a9961ed7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:07.449183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d26d9d4e-e1a8-4ef8-980b-38c8a9961ed7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:15:07.449183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:07.449183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:07.449183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:07.449183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:07.449183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:07.449183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:07.449183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:07.449183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:07.449183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:07.449183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:07.449183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:07.449183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:07.449183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:07.449183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:07.449183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:07.450141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:07.450141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:07.450141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:07.450141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:07.450141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:07.450141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:07.450141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:07.450141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:07.450141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:07.450141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:07.450141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:07.450141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:07.450141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:07.450141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:07.450141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:07.450141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:07.450937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:07.450937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:07.450937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:07.450937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:07.450937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:07.450937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:07.450937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:07.450937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:07.450937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:07.450937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:07.450937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:15:07.450937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:07.450937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 72/144] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:07.457607 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ceadae-53fb-48a8-9198-4b77fa12767a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:07.574521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ceadae-53fb-48a8-9198-4b77fa12767a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:07.604636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:07.610933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b3d25-7b21-400c-959a-b5381539f72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:07.611872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b3d25-7b21-400c-959a-b5381539f72e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:07.612397 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b3d25-7b21-400c-959a-b5381539f72e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:07.612749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b3d25-7b21-400c-959a-b5381539f72e None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:07.613881 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b3d25-7b21-400c-959a-b5381539f72e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:07.614237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b3d25-7b21-400c-959a-b5381539f72e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:07.615714 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b3d25-7b21-400c-959a-b5381539f72e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:07.625563 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 73/145] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:07.630984 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d9d4e-e1a8-4ef8-980b-38c8a9961ed7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:07.763078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d9d4e-e1a8-4ef8-980b-38c8a9961ed7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:07.793505 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:07.803115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168a42b-d06d-4499-9247-84ff74b5c1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:07.804617 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168a42b-d06d-4499-9247-84ff74b5c1fe None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:07.804903 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168a42b-d06d-4499-9247-84ff74b5c1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:07.805178 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168a42b-d06d-4499-9247-84ff74b5c1fe None admin] PATH_INFO: `/v3/users/cc013382f7504c65a526dd7f68c9e78f` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:07.806453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168a42b-d06d-4499-9247-84ff74b5c1fe None admin] RBAC: Authorizing `identity:get_user(user_id=cc013382f7504c65a526dd7f68c9e78f)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:07.807961 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168a42b-d06d-4499-9247-84ff74b5c1fe None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:07.809797 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 73/146] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:07 2022] GET /identity/v3/users/cc013382f7504c65a526dd7f68c9e78f => generated 289 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:07.815579 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b3d25-7b21-400c-959a-b5381539f72e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:07.921715 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b3d25-7b21-400c-959a-b5381539f72e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:07.960258 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33438612-ad41-441e-83ea-de715ac5df97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:07.961171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33438612-ad41-441e-83ea-de715ac5df97 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:07.961448 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33438612-ad41-441e-83ea-de715ac5df97 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:07.961715 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33438612-ad41-441e-83ea-de715ac5df97 None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:07.963338 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33438612-ad41-441e-83ea-de715ac5df97 None admin] RBAC: Authorizing `identity:get_project(project_id=09d730b36d8f4ab482dc9ed98757bb0b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:07.964667 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33438612-ad41-441e-83ea-de715ac5df97 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:07.966400 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 74/147] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:07 2022] GET /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:07.972382 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168a42b-d06d-4499-9247-84ff74b5c1fe None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:08.085932 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168a42b-d06d-4499-9247-84ff74b5c1fe None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:08.122815 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb6424-ebf4-4bd2-83a7-5581461383df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:08.123708 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb6424-ebf4-4bd2-83a7-5581461383df None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:08.124006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb6424-ebf4-4bd2-83a7-5581461383df None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:08.124271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb6424-ebf4-4bd2-83a7-5581461383df None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b/users/cc013382f7504c65a526dd7f68c9e78f/roles/e6c6a363f7034bd0a72b72ffe324aab8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:08.126276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb6424-ebf4-4bd2-83a7-5581461383df None admin] RBAC: Authorizing `identity:create_grant(project_id=09d730b36d8f4ab482dc9ed98757bb0b, user_id=cc013382f7504c65a526dd7f68c9e78f, role_id=e6c6a363f7034bd0a72b72ffe324aab8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:08.127739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb6424-ebf4-4bd2-83a7-5581461383df None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:08.155530 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 74/148] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:15:07 2022] PUT /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b/users/cc013382f7504c65a526dd7f68c9e78f/roles/e6c6a363f7034bd0a72b72ffe324aab8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:15:09.257132 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1584a-6cce-429c-adf8-a0af7951e126 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:09.257635 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1584a-6cce-429c-adf8-a0af7951e126 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:09.257710 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1584a-6cce-429c-adf8-a0af7951e126 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:09.259298 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 75/149] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:09.264745 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32636633-c5bc-47c2-9941-7c55808c0cfb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:09.265174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32636633-c5bc-47c2-9941-7c55808c0cfb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:09.265353 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32636633-c5bc-47c2-9941-7c55808c0cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:09.289658 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-32636633-c5bc-47c2-9941-7c55808c0cfb None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:09.290886 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-32636633-c5bc-47c2-9941-7c55808c0cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:09.297869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f731b1-7cdf-483b-b779-139817432aed None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:09.298162 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f731b1-7cdf-483b-b779-139817432aed None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:09.298444 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f731b1-7cdf-483b-b779-139817432aed None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:09.300032 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 76/150] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:09.305127 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3578142-77a1-4610-bb0c-c23a9ccb1591 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:09.305474 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3578142-77a1-4610-bb0c-c23a9ccb1591 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:09.305759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3578142-77a1-4610-bb0c-c23a9ccb1591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:09.322800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d3578142-77a1-4610-bb0c-c23a9ccb1591 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:09.324022 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d3578142-77a1-4610-bb0c-c23a9ccb1591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:09.363219 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:09.378034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 75/151] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:09.386410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:09.398490 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 77/152] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:09.405919 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32636633-c5bc-47c2-9941-7c55808c0cfb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:09.421671 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3578142-77a1-4610-bb0c-c23a9ccb1591 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:09.512098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32636633-c5bc-47c2-9941-7c55808c0cfb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:09.526692 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3578142-77a1-4610-bb0c-c23a9ccb1591 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:09.549507 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441caca-6537-4e64-bfdb-987894be31f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:09.550642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441caca-6537-4e64-bfdb-987894be31f3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:09.551175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441caca-6537-4e64-bfdb-987894be31f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:09.551732 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441caca-6537-4e64-bfdb-987894be31f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:09.562610 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978b281-079e-4eaf-adda-cc538c8410e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:09.563420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978b281-079e-4eaf-adda-cc538c8410e9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:09.563534 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2441caca-6537-4e64-bfdb-987894be31f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:09.563698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978b281-079e-4eaf-adda-cc538c8410e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:09.563975 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978b281-079e-4eaf-adda-cc538c8410e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:09.572888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-2441caca-6537-4e64-bfdb-987894be31f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:15:09.572888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:09.572888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:09.572888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:09.572888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:09.572888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:09.572888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:09.572888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:09.572888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:09.572888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:09.572888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:09.572888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:09.572888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:09.572888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:09.572888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:09.572888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:09.573780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:09.573780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:09.573780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:09.573780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:09.573780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:09.573780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:09.573780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:09.573780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:09.573780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:09.573780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:09.573780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:09.573780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:09.573780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:09.573780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:09.573780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:09.573780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:09.574549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:09.574549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:09.574549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:09.574549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:09.574549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:09.574549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:09.574549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:09.574549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:09.574549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:09.574549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:09.574549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:15:09.574549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:09.575617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2978b281-079e-4eaf-adda-cc538c8410e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:09.577466 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 76/153] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:15:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:09.583185 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441caca-6537-4e64-bfdb-987894be31f3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:09.583411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-2978b281-079e-4eaf-adda-cc538c8410e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:09.583411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:09.583411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:09.583411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:09.583411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:09.583411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:09.583411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:09.583411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:09.583411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:09.583411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:09.583411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:09.583411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:09.583411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:09.583411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:09.583411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:09.583411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:09.584246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:09.584246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:09.584246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:09.584246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:09.584246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:09.584246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:09.584246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:09.584246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:09.584246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:09.584246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:09.584246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:09.584246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:09.584246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:09.584246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:09.584246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:09.584246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:09.585619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:09.585619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:09.585619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:09.585619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:09.585619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:09.585619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:09.585619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:09.585619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:09.585619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:09.585619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:09.585619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:09.585619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:09.589029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 78/154] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:09.591704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978b281-079e-4eaf-adda-cc538c8410e9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:09.646372 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:09.651554 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:09.701054 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441caca-6537-4e64-bfdb-987894be31f3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:09.716827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978b281-079e-4eaf-adda-cc538c8410e9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:09.737785 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd1b42-67b8-42bf-8861-43f649e31cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:09.738899 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd1b42-67b8-42bf-8861-43f649e31cfd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:09.739395 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd1b42-67b8-42bf-8861-43f649e31cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:09.739985 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd1b42-67b8-42bf-8861-43f649e31cfd None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:09.741084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd1b42-67b8-42bf-8861-43f649e31cfd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:09.741616 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd1b42-67b8-42bf-8861-43f649e31cfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:09.743214 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd1b42-67b8-42bf-8861-43f649e31cfd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:09.757383 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 77/155] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:09.762168 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd1b42-67b8-42bf-8861-43f649e31cfd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:09.771583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fb6c9-678c-49a7-af12-20e99cf8a498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:09.773136 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fb6c9-678c-49a7-af12-20e99cf8a498 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:09.778438 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fb6c9-678c-49a7-af12-20e99cf8a498 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:09.778967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fb6c9-678c-49a7-af12-20e99cf8a498 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:09.779969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fb6c9-678c-49a7-af12-20e99cf8a498 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:09.780468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fb6c9-678c-49a7-af12-20e99cf8a498 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:09.782075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fb6c9-678c-49a7-af12-20e99cf8a498 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:09.799070 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 79/156] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:09.805221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fb6c9-678c-49a7-af12-20e99cf8a498 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:09.965566 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd1b42-67b8-42bf-8861-43f649e31cfd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:09.991189 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:10.020570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e6a77-48d2-46b8-a155-8371102abe48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:10.023237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fb6c9-678c-49a7-af12-20e99cf8a498 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:10.024333 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e6a77-48d2-46b8-a155-8371102abe48 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:10.024827 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e6a77-48d2-46b8-a155-8371102abe48 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:10.025314 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e6a77-48d2-46b8-a155-8371102abe48 None admin] PATH_INFO: `/v3/users/06ec5a62dde8447fa4c9a00a0f1828f8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:10.028051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e6a77-48d2-46b8-a155-8371102abe48 None admin] RBAC: Authorizing `identity:get_user(user_id=06ec5a62dde8447fa4c9a00a0f1828f8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:10.029744 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e6a77-48d2-46b8-a155-8371102abe48 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:10.031809 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 78/157] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:09 2022] GET /identity/v3/users/06ec5a62dde8447fa4c9a00a0f1828f8 => generated 247 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:10.037832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e6a77-48d2-46b8-a155-8371102abe48 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:10.060935 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:10.101650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc595b3-a495-42d8-8134-ba4d705b090a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:10.103131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc595b3-a495-42d8-8134-ba4d705b090a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:10.106532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc595b3-a495-42d8-8134-ba4d705b090a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:10.107029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc595b3-a495-42d8-8134-ba4d705b090a None admin] PATH_INFO: `/v3/users/4eecd6829d5a4a319fa4af969eb3d39d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:10.108530 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc595b3-a495-42d8-8134-ba4d705b090a None admin] RBAC: Authorizing `identity:get_user(user_id=4eecd6829d5a4a319fa4af969eb3d39d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:10.111693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc595b3-a495-42d8-8134-ba4d705b090a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:10.114430 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 80/158] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:09 2022] GET /identity/v3/users/4eecd6829d5a4a319fa4af969eb3d39d => generated 275 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:10.120283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc595b3-a495-42d8-8134-ba4d705b090a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:10.179890 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e6a77-48d2-46b8-a155-8371102abe48 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:10.216576 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856c471-92d4-4138-b134-20679aef1858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:10.217565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856c471-92d4-4138-b134-20679aef1858 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:10.218045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856c471-92d4-4138-b134-20679aef1858 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:10.218510 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856c471-92d4-4138-b134-20679aef1858 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:10.267573 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:10.294115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856c471-92d4-4138-b134-20679aef1858 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:10.295942 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856c471-92d4-4138-b134-20679aef1858 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:10.386833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc595b3-a495-42d8-8134-ba4d705b090a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:10.402384 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 79/159] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:10.404102 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bfe9b-2202-4163-8abf-8206ae330419 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:10.404630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bfe9b-2202-4163-8abf-8206ae330419 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:10.405111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bfe9b-2202-4163-8abf-8206ae330419 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:10.406851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 80/160] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:10.413642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bfe9b-2202-4163-8abf-8206ae330419 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:10.475062 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a02b5-3270-4217-be1b-cf50c5c2d9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:10.475728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a02b5-3270-4217-be1b-cf50c5c2d9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:10.476023 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a02b5-3270-4217-be1b-cf50c5c2d9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:10.476304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a02b5-3270-4217-be1b-cf50c5c2d9c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:10.552561 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:10.568121 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:10.579672 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a02b5-3270-4217-be1b-cf50c5c2d9c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:10.581378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a02b5-3270-4217-be1b-cf50c5c2d9c6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:10.631523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bfe9b-2202-4163-8abf-8206ae330419 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:10.697982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 81/161] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 10:15:10.700083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561daa2-08f1-42d7-9fc7-bab454f72861 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:10.700520 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561daa2-08f1-42d7-9fc7-bab454f72861 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:10.700903 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561daa2-08f1-42d7-9fc7-bab454f72861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:10.709788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd88ed4-7f2a-43dd-b3cd-5e673b26c783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:10.710767 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd88ed4-7f2a-43dd-b3cd-5e673b26c783 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:10.711060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd88ed4-7f2a-43dd-b3cd-5e673b26c783 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:10.711338 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd88ed4-7f2a-43dd-b3cd-5e673b26c783 None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:10.712570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd88ed4-7f2a-43dd-b3cd-5e673b26c783 None admin] RBAC: Authorizing `identity:get_project(project_id=09d730b36d8f4ab482dc9ed98757bb0b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:10.714050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd88ed4-7f2a-43dd-b3cd-5e673b26c783 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:10.715771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 81/162] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:10 2022] GET /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:10.716976 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee8b98-4d81-4c8f-9776-6076193400bc None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:10.717262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee8b98-4d81-4c8f-9776-6076193400bc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:10.717597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee8b98-4d81-4c8f-9776-6076193400bc None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:10.719521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3561daa2-08f1-42d7-9fc7-bab454f72861 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:10.720966 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3561daa2-08f1-42d7-9fc7-bab454f72861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:10.724766 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 82/163] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:10.726913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee8b98-4d81-4c8f-9776-6076193400bc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:10.862797 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 82/164] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:10.863751 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561daa2-08f1-42d7-9fc7-bab454f72861 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:10.892026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:10.946034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee8b98-4d81-4c8f-9776-6076193400bc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:11.019357 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561daa2-08f1-42d7-9fc7-bab454f72861 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:11.021250 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695193eb-be51-4be6-aa21-4c7f1cf15346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:11.022144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695193eb-be51-4be6-aa21-4c7f1cf15346 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:11.023230 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695193eb-be51-4be6-aa21-4c7f1cf15346 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:11.023522 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695193eb-be51-4be6-aa21-4c7f1cf15346 None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:11.025003 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695193eb-be51-4be6-aa21-4c7f1cf15346 None admin] RBAC: Authorizing `identity:get_project(project_id=09d730b36d8f4ab482dc9ed98757bb0b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:11.026520 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695193eb-be51-4be6-aa21-4c7f1cf15346 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:11.028622 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 83/165] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:10 2022] GET /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:11.029670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d0cd1-e701-4f56-a14b-71b1bcbd42d8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:11.029956 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d0cd1-e701-4f56-a14b-71b1bcbd42d8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:11.030235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d0cd1-e701-4f56-a14b-71b1bcbd42d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:11.063073 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fc8d0cd1-e701-4f56-a14b-71b1bcbd42d8 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:11.064803 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fc8d0cd1-e701-4f56-a14b-71b1bcbd42d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:11.074498 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:11.102106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b09c4-8e1d-49d6-93df-34b427071af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:11.102871 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b09c4-8e1d-49d6-93df-34b427071af7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:11.103498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b09c4-8e1d-49d6-93df-34b427071af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:11.103843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b09c4-8e1d-49d6-93df-34b427071af7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:11.104782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b09c4-8e1d-49d6-93df-34b427071af7 None admin] RBAC: Adding query filter params (role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b, user.id=06ec5a62dde8447fa4c9a00a0f1828f8) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:11.105182 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b09c4-8e1d-49d6-93df-34b427071af7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:11.112526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b09c4-8e1d-49d6-93df-34b427071af7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:11.121330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 83/166] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:10 2022] GET /identity/v3/role_assignments?user.id=06ec5a62dde8447fa4c9a00a0f1828f8&scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:11.122204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b09c4-8e1d-49d6-93df-34b427071af7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:11.179545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 84/167] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:11.179857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f48d3-5666-4422-aec1-d7b361d79871 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:11.180109 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f48d3-5666-4422-aec1-d7b361d79871 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:11.180388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f48d3-5666-4422-aec1-d7b361d79871 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:11.186378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 85/168] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:11.187326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f48d3-5666-4422-aec1-d7b361d79871 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:11.273827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:11.339536 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b09c4-8e1d-49d6-93df-34b427071af7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:11.386534 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f48d3-5666-4422-aec1-d7b361d79871 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:11.420807 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d458b-7fba-4ba9-b43d-2e16c6a65c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:11.420807 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d458b-7fba-4ba9-b43d-2e16c6a65c90 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:11.420807 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d458b-7fba-4ba9-b43d-2e16c6a65c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:11.420807 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d458b-7fba-4ba9-b43d-2e16c6a65c90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:11.440434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d458b-7fba-4ba9-b43d-2e16c6a65c90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:11.441358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92772c15-1bd8-4ffc-b733-ced214addcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:11.442274 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92772c15-1bd8-4ffc-b733-ced214addcbf None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:11.442755 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92772c15-1bd8-4ffc-b733-ced214addcbf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:11.443228 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92772c15-1bd8-4ffc-b733-ced214addcbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:11.444332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92772c15-1bd8-4ffc-b733-ced214addcbf None admin] RBAC: Adding query filter params (role.id=2820090979cb487db491f835e03e6471, scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b, user.id=4eecd6829d5a4a319fa4af969eb3d39d) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:11.445384 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92772c15-1bd8-4ffc-b733-ced214addcbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:11.447039 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92772c15-1bd8-4ffc-b733-ced214addcbf None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:11.453872 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 86/169] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:11 2022] GET /identity/v3/role_assignments?user.id=4eecd6829d5a4a319fa4af969eb3d39d&scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b&role.id=2820090979cb487db491f835e03e6471 => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:11.455474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b4f52-755d-4df4-9bed-858d4ce7fe9a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:11.455970 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b4f52-755d-4df4-9bed-858d4ce7fe9a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:11.456468 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b4f52-755d-4df4-9bed-858d4ce7fe9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:11.457178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-503d458b-7fba-4ba9-b43d-2e16c6a65c90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:11.457178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:11.457178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:11.457178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:11.457178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:11.457178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:11.457178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:11.457178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:11.457178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:11.457178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:11.457178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:11.457178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:11.457178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:11.457178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:11.457178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:11.457178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:11.458106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:11.458106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:11.458106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:11.458106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:11.458106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:11.458106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:11.458106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:11.458106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:11.458106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:11.458106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:11.458106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:11.458106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:11.458106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:11.458106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:11.458106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:11.458106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:11.458901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:11.458901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:11.458901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:11.458901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:11.458901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:11.458901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:11.458901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:11.458901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:11.458901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:11.458901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:11.458901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:11.458901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:11.458901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 84/170] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:11.460067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d458b-7fba-4ba9-b43d-2e16c6a65c90 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:11.476903 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:11.479613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-798b4f52-755d-4df4-9bed-858d4ce7fe9a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:11.481139 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-798b4f52-755d-4df4-9bed-858d4ce7fe9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:11.585305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 87/171] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:11.586116 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b4f52-755d-4df4-9bed-858d4ce7fe9a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:11.649823 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d458b-7fba-4ba9-b43d-2e16c6a65c90 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:11.707447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b4f52-755d-4df4-9bed-858d4ce7fe9a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:11.747577 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e82867-192a-4a57-98d5-38b7026b67e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:11.748280 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e82867-192a-4a57-98d5-38b7026b67e6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:11.748593 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e82867-192a-4a57-98d5-38b7026b67e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:11.748910 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e82867-192a-4a57-98d5-38b7026b67e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:11.749806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e82867-192a-4a57-98d5-38b7026b67e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:11.750103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc77c2-8c6a-4cac-8f55-19b7f1982989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:11.750310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e82867-192a-4a57-98d5-38b7026b67e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:11.751353 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc77c2-8c6a-4cac-8f55-19b7f1982989 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:11.751595 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e82867-192a-4a57-98d5-38b7026b67e6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:11.752047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc77c2-8c6a-4cac-8f55-19b7f1982989 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:11.752537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc77c2-8c6a-4cac-8f55-19b7f1982989 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:11.758898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:11.760469 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:11.762976 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 88/172] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:11.763980 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e82867-192a-4a57-98d5-38b7026b67e6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:11.781553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc77c2-8c6a-4cac-8f55-19b7f1982989 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:11.797952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-33bc77c2-8c6a-4cac-8f55-19b7f1982989 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:11.797952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:11.797952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:11.797952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:11.797952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:11.797952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:11.797952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:11.797952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:11.797952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:11.797952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:11.797952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:11.797952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:11.797952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:11.797952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:11.797952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:11.797952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:11.798933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:11.798933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:11.798933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:11.798933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:11.798933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:11.798933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:11.798933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:11.798933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:11.798933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:11.798933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:11.798933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:11.798933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:11.798933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:11.798933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:11.798933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:11.798933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:11.799716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:11.799716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:11.799716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:11.799716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:11.799716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:11.799716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:11.799716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:11.799716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:11.799716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:11.799716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:11.799716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:11.799716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:11.799716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 85/173] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:15:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:11.800402 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc77c2-8c6a-4cac-8f55-19b7f1982989 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:11.890608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e82867-192a-4a57-98d5-38b7026b67e6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:11.926185 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91988f8b-34a8-45ad-a11e-86b0447ebaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:11.927000 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91988f8b-34a8-45ad-a11e-86b0447ebaeb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:11.927289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91988f8b-34a8-45ad-a11e-86b0447ebaeb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:11.927571 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91988f8b-34a8-45ad-a11e-86b0447ebaeb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:11.940609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91988f8b-34a8-45ad-a11e-86b0447ebaeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:11.949379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-91988f8b-34a8-45ad-a11e-86b0447ebaeb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:15:11.949379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:11.949379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:11.949379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:11.949379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:11.949379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:11.949379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:11.949379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:11.949379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:11.949379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:11.949379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:11.949379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:11.949379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:11.949379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:11.949379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:11.949379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:11.950401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:11.950401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:11.950401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:11.950401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:11.950401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:11.950401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:11.950401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:11.950401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:11.950401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:11.950401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:11.950401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:11.950401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:11.950401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:11.950401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:11.950401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:11.950401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:11.951184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:11.951184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:11.951184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:11.951184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:11.951184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:11.951184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:11.951184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:11.951184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:11.951184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:11.951184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:11.951184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:15:11.951184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:11.951184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 89/174] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:15:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:11.951845 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91988f8b-34a8-45ad-a11e-86b0447ebaeb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:11.976554 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc77c2-8c6a-4cac-8f55-19b7f1982989 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:12.033428 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:12.034613 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12085b98-1fd4-4cff-9d24-326038b2167d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:12.035513 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12085b98-1fd4-4cff-9d24-326038b2167d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:12.035799 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12085b98-1fd4-4cff-9d24-326038b2167d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:12.036078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12085b98-1fd4-4cff-9d24-326038b2167d None admin] PATH_INFO: `/v3/users/4eecd6829d5a4a319fa4af969eb3d39d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:12.041298 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12085b98-1fd4-4cff-9d24-326038b2167d None admin] RBAC: Authorizing `identity:get_user(user_id=4eecd6829d5a4a319fa4af969eb3d39d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:12.042793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12085b98-1fd4-4cff-9d24-326038b2167d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:12.044521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 86/175] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:11 2022] GET /identity/v3/users/4eecd6829d5a4a319fa4af969eb3d39d => generated 275 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:12.045305 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12085b98-1fd4-4cff-9d24-326038b2167d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:12.068898 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91988f8b-34a8-45ad-a11e-86b0447ebaeb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:12.106028 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574dd38-e81f-48a5-9bc4-d8b985dd5663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:12.106804 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574dd38-e81f-48a5-9bc4-d8b985dd5663 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:12.107097 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574dd38-e81f-48a5-9bc4-d8b985dd5663 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:12.107379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574dd38-e81f-48a5-9bc4-d8b985dd5663 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:12.108143 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574dd38-e81f-48a5-9bc4-d8b985dd5663 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:12.108454 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574dd38-e81f-48a5-9bc4-d8b985dd5663 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:12.109979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574dd38-e81f-48a5-9bc4-d8b985dd5663 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:12.119638 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 90/176] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:15:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:12.120452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574dd38-e81f-48a5-9bc4-d8b985dd5663 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:12.145741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:12.231441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12085b98-1fd4-4cff-9d24-326038b2167d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:12.231658 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574dd38-e81f-48a5-9bc4-d8b985dd5663 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:12.257552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:12.270854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ab481-26a0-4c7b-8c7e-4f8be193a788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:12.271630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ab481-26a0-4c7b-8c7e-4f8be193a788 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:12.271918 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ab481-26a0-4c7b-8c7e-4f8be193a788 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:12.272188 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ab481-26a0-4c7b-8c7e-4f8be193a788 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:12.297510 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5277b-d5b2-435e-bfb3-4c609197b45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:12.298193 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5277b-d5b2-435e-bfb3-4c609197b45e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:12.298651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5277b-d5b2-435e-bfb3-4c609197b45e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:12.299072 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5277b-d5b2-435e-bfb3-4c609197b45e None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:12.299980 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5277b-d5b2-435e-bfb3-4c609197b45e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:12.300448 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5277b-d5b2-435e-bfb3-4c609197b45e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:12.302214 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5277b-d5b2-435e-bfb3-4c609197b45e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:12.319994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 87/177] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:12.321108 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5277b-d5b2-435e-bfb3-4c609197b45e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:12.345264 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ab481-26a0-4c7b-8c7e-4f8be193a788 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:12.347114 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ab481-26a0-4c7b-8c7e-4f8be193a788 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:12.439211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5277b-d5b2-435e-bfb3-4c609197b45e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:12.454020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 91/178] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:12.455457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ab481-26a0-4c7b-8c7e-4f8be193a788 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:12.466473 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:12.478611 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38d4d6-5997-400b-a13d-2cae1119516e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:12.479081 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:12.479492 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38d4d6-5997-400b-a13d-2cae1119516e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:12.479782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38d4d6-5997-400b-a13d-2cae1119516e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:12.480065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38d4d6-5997-400b-a13d-2cae1119516e None admin] PATH_INFO: `/v3/users/4eecd6829d5a4a319fa4af969eb3d39d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:12.481494 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38d4d6-5997-400b-a13d-2cae1119516e None admin] RBAC: Authorizing `identity:get_user(user_id=4eecd6829d5a4a319fa4af969eb3d39d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:12.483078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38d4d6-5997-400b-a13d-2cae1119516e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:12.485204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 88/179] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:12 2022] GET /identity/v3/users/4eecd6829d5a4a319fa4af969eb3d39d => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:12.486489 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ab8e7-e898-4e0c-a740-ce732d30481c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:12.486786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ab8e7-e898-4e0c-a740-ce732d30481c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:12.487240 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ab8e7-e898-4e0c-a740-ce732d30481c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:12.488445 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 89/180] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:12.489639 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0096ce-2822-42bd-9ca4-0c2a5b58c898 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:12.490190 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0096ce-2822-42bd-9ca4-0c2a5b58c898 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:12.490190 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0096ce-2822-42bd-9ca4-0c2a5b58c898 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:12.491645 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 90/181] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:12.496150 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0096ce-2822-42bd-9ca4-0c2a5b58c898 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:12.561629 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ab481-26a0-4c7b-8c7e-4f8be193a788 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:12.598605 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ae23d-a255-45ec-a0f8-8a0f475c5444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:12.599469 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ae23d-a255-45ec-a0f8-8a0f475c5444 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:12.599750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ae23d-a255-45ec-a0f8-8a0f475c5444 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:12.600023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ae23d-a255-45ec-a0f8-8a0f475c5444 None admin] PATH_INFO: `/v3/users/b8bc8c0a6f2640f69272813cdd67ca2e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:12.610528 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0096ce-2822-42bd-9ca4-0c2a5b58c898 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:12.618247 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ae23d-a255-45ec-a0f8-8a0f475c5444 None admin] RBAC: Authorizing `identity:get_user(user_id=b8bc8c0a6f2640f69272813cdd67ca2e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:12.619745 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ae23d-a255-45ec-a0f8-8a0f475c5444 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:12.621685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 92/182] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:12 2022] GET /identity/v3/users/b8bc8c0a6f2640f69272813cdd67ca2e => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:12.622703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19658de5-d9ae-47ca-873f-20628a514650 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:12.622989 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19658de5-d9ae-47ca-873f-20628a514650 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:12.623266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19658de5-d9ae-47ca-873f-20628a514650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:12.639592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-19658de5-d9ae-47ca-873f-20628a514650 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:12.641037 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-19658de5-d9ae-47ca-873f-20628a514650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:12.651692 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc60ca-dc2e-4e9c-8501-720e27833d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:12.652589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc60ca-dc2e-4e9c-8501-720e27833d9f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:12.652880 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc60ca-dc2e-4e9c-8501-720e27833d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:12.653158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc60ca-dc2e-4e9c-8501-720e27833d9f None admin] PATH_INFO: `/v3/projects/bcfc47788c5742db9a4a501fd70663de` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:12.654345 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc60ca-dc2e-4e9c-8501-720e27833d9f None admin] RBAC: Authorizing `identity:get_project(project_id=bcfc47788c5742db9a4a501fd70663de)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:12.655789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc60ca-dc2e-4e9c-8501-720e27833d9f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:12.657537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 91/183] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:12 2022] GET /identity/v3/projects/bcfc47788c5742db9a4a501fd70663de => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:12.658309 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc60ca-dc2e-4e9c-8501-720e27833d9f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:12.706050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 93/184] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:12.707330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2776d42-92e1-4635-bd87-694700b5f884 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:12.707628 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2776d42-92e1-4635-bd87-694700b5f884 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:12.707911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2776d42-92e1-4635-bd87-694700b5f884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:12.724028 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f2776d42-92e1-4635-bd87-694700b5f884 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:12.725531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f2776d42-92e1-4635-bd87-694700b5f884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:12.726129 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:12.727533 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:12.768833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc60ca-dc2e-4e9c-8501-720e27833d9f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:12.793082 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 94/185] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:12.794384 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f39747-c33d-4617-aa7d-e2f3c84c7426 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:12.794670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f39747-c33d-4617-aa7d-e2f3c84c7426 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:12.794946 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f39747-c33d-4617-aa7d-e2f3c84c7426 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:12.796206 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 95/186] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:12.796917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f39747-c33d-4617-aa7d-e2f3c84c7426 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:12.808766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de93a38-22e0-4569-a08f-6d5a950ca33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:12.809424 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de93a38-22e0-4569-a08f-6d5a950ca33e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:12.809703 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de93a38-22e0-4569-a08f-6d5a950ca33e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:12.809980 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de93a38-22e0-4569-a08f-6d5a950ca33e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:12.875180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de93a38-22e0-4569-a08f-6d5a950ca33e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:12.876695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de93a38-22e0-4569-a08f-6d5a950ca33e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:12.899251 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f39747-c33d-4617-aa7d-e2f3c84c7426 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:12.932485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:12.933645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5778dd6-00dc-449b-98ab-adf0adde07fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:12.934291 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5778dd6-00dc-449b-98ab-adf0adde07fc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:12.934565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5778dd6-00dc-449b-98ab-adf0adde07fc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:12.934859 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5778dd6-00dc-449b-98ab-adf0adde07fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:12.943930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:12.974677 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 92/187] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:12.975258 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de93a38-22e0-4569-a08f-6d5a950ca33e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:13.004147 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5778dd6-00dc-449b-98ab-adf0adde07fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:13.005668 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5778dd6-00dc-449b-98ab-adf0adde07fc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:13.075209 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de93a38-22e0-4569-a08f-6d5a950ca33e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:13.102757 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 96/188] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:13.103852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5778dd6-00dc-449b-98ab-adf0adde07fc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:13.109034 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec932b-c319-4c3f-b424-47b079eaeb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:13.109689 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec932b-c319-4c3f-b424-47b079eaeb2e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:13.109975 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec932b-c319-4c3f-b424-47b079eaeb2e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:13.110253 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec932b-c319-4c3f-b424-47b079eaeb2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:13.111110 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec932b-c319-4c3f-b424-47b079eaeb2e None admin] RBAC: Adding query filter params (role.id=2820090979cb487db491f835e03e6471, scope.project.id=bcfc47788c5742db9a4a501fd70663de, user.id=4eecd6829d5a4a319fa4af969eb3d39d) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:13.111543 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec932b-c319-4c3f-b424-47b079eaeb2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:13.113028 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec932b-c319-4c3f-b424-47b079eaeb2e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:13.119727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 93/189] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:12 2022] GET /identity/v3/role_assignments?user.id=4eecd6829d5a4a319fa4af969eb3d39d&scope.project.id=bcfc47788c5742db9a4a501fd70663de&role.id=2820090979cb487db491f835e03e6471 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:13.120916 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79110f65-0c22-4fd2-9a1b-ac07506ac56f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:13.121204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79110f65-0c22-4fd2-9a1b-ac07506ac56f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:13.121480 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79110f65-0c22-4fd2-9a1b-ac07506ac56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:13.144543 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:13.151809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-79110f65-0c22-4fd2-9a1b-ac07506ac56f None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:13.153092 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-79110f65-0c22-4fd2-9a1b-ac07506ac56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:13.174075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:13.225302 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5778dd6-00dc-449b-98ab-adf0adde07fc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:13.236551 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 94/190] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:13.236551 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79110f65-0c22-4fd2-9a1b-ac07506ac56f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:13.276759 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f0047-6bd0-469e-896f-edcee066bc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:13.277577 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f0047-6bd0-469e-896f-edcee066bc5c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:13.277870 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f0047-6bd0-469e-896f-edcee066bc5c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:13.278760 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f0047-6bd0-469e-896f-edcee066bc5c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:13.292205 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f0047-6bd0-469e-896f-edcee066bc5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:13.300441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-349f0047-6bd0-469e-896f-edcee066bc5c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:13.300441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:13.300441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:13.300441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:13.300441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:13.300441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:13.300441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:13.300441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:13.300441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:13.300441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:13.300441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:13.300441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:13.300441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:13.300441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:13.300441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:13.300441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:13.301379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:13.301379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:13.301379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:13.301379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:13.301379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:13.301379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:13.301379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:13.301379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:13.301379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:13.301379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:13.301379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:13.301379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:13.301379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:13.301379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:13.301379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:13.301379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:13.302153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:13.302153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:13.302153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:13.302153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:13.302153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:13.302153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:13.302153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:13.302153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:13.302153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:13.302153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:13.302153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:13.302153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:13.302153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 97/191] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:15:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:13.302790 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f0047-6bd0-469e-896f-edcee066bc5c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:13.351666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79110f65-0c22-4fd2-9a1b-ac07506ac56f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:13.386940 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dbdfeb-404b-4b89-8bd8-93fe9cea4656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:13.387735 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbdfeb-404b-4b89-8bd8-93fe9cea4656 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:13.388017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbdfeb-404b-4b89-8bd8-93fe9cea4656 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:13.388294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbdfeb-404b-4b89-8bd8-93fe9cea4656 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:13.389475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dbdfeb-404b-4b89-8bd8-93fe9cea4656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:13.390927 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dbdfeb-404b-4b89-8bd8-93fe9cea4656 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:13.392561 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 95/192] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:13.393311 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dbdfeb-404b-4b89-8bd8-93fe9cea4656 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:13.411230 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:13.415277 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f0047-6bd0-469e-896f-edcee066bc5c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:13.416594 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:13.449715 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9141f-4147-45f7-9ef5-050577f364a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:13.450566 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9141f-4147-45f7-9ef5-050577f364a1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:13.450851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9141f-4147-45f7-9ef5-050577f364a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:13.451125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9141f-4147-45f7-9ef5-050577f364a1 None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:13.452296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9141f-4147-45f7-9ef5-050577f364a1 None admin] RBAC: Authorizing `identity:get_project(project_id=09d730b36d8f4ab482dc9ed98757bb0b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:13.453789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e9141f-4147-45f7-9ef5-050577f364a1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:13.455482 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 98/193] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:13 2022] GET /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:13.456241 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9141f-4147-45f7-9ef5-050577f364a1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:13.496167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dbdfeb-404b-4b89-8bd8-93fe9cea4656 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:13.533903 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ba275-3a36-4cac-b03b-44f34afafa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:13.534768 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ba275-3a36-4cac-b03b-44f34afafa24 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:13.535055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ba275-3a36-4cac-b03b-44f34afafa24 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:13.535336 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ba275-3a36-4cac-b03b-44f34afafa24 None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:13.536779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ba275-3a36-4cac-b03b-44f34afafa24 None admin] RBAC: Authorizing `identity:get_project(project_id=57b2ea019ba3413bafb7714839f85a22)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:13.538892 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ba275-3a36-4cac-b03b-44f34afafa24 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:13.540621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 96/194] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:13 2022] GET /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:13.541388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ba275-3a36-4cac-b03b-44f34afafa24 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:13.557476 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9141f-4147-45f7-9ef5-050577f364a1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:13.592939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12200b9-7a42-483b-8c00-d5deaf774a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:13.593732 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12200b9-7a42-483b-8c00-d5deaf774a06 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:13.594020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12200b9-7a42-483b-8c00-d5deaf774a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:13.594295 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12200b9-7a42-483b-8c00-d5deaf774a06 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:13.605224 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12200b9-7a42-483b-8c00-d5deaf774a06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:13.612924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c12200b9-7a42-483b-8c00-d5deaf774a06 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:15:13.612924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:13.612924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:13.612924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:13.612924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:13.612924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:13.612924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:13.612924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:13.612924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:13.612924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:13.612924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:13.612924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:13.612924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:13.612924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:13.612924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:13.612924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:13.613799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:13.613799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:13.613799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:13.613799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:13.613799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:13.613799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:13.613799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:13.613799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:13.613799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:13.613799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:13.613799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:13.613799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:13.613799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:13.613799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:13.613799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:13.613799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:13.614621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:13.614621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:13.614621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:13.614621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:13.614621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:13.614621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:13.614621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:13.614621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:13.614621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:13.614621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:13.614621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:15:13.614621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:13.614621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 99/195] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:13.615271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12200b9-7a42-483b-8c00-d5deaf774a06 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:13.633906 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:13.643960 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ba275-3a36-4cac-b03b-44f34afafa24 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:13.644843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:13.679295 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc2ea9-c200-42c3-85b2-6e20fe1c218b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:13.679964 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc2ea9-c200-42c3-85b2-6e20fe1c218b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:13.680251 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc2ea9-c200-42c3-85b2-6e20fe1c218b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:13.680556 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc2ea9-c200-42c3-85b2-6e20fe1c218b None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:13.681316 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc2ea9-c200-42c3-85b2-6e20fe1c218b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:13.681628 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc2ea9-c200-42c3-85b2-6e20fe1c218b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:13.683014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc2ea9-c200-42c3-85b2-6e20fe1c218b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:13.691405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 97/196] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:15:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:13.692191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc2ea9-c200-42c3-85b2-6e20fe1c218b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:13.717742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12200b9-7a42-483b-8c00-d5deaf774a06 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:13.750553 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79d847-5c85-4466-a3ca-e3370d809376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:13.751152 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79d847-5c85-4466-a3ca-e3370d809376 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:13.751434 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79d847-5c85-4466-a3ca-e3370d809376 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:13.751713 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79d847-5c85-4466-a3ca-e3370d809376 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:13.752424 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79d847-5c85-4466-a3ca-e3370d809376 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:13.753898 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79d847-5c85-4466-a3ca-e3370d809376 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:13.773997 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 100/197] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 18 10:15:13 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:15:13.774906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79d847-5c85-4466-a3ca-e3370d809376 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:13.812485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc2ea9-c200-42c3-85b2-6e20fe1c218b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:13.852207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426f013-99ed-4766-bdda-d706f0240561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:13.853424 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426f013-99ed-4766-bdda-d706f0240561 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:13.853628 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426f013-99ed-4766-bdda-d706f0240561 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:13.853971 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426f013-99ed-4766-bdda-d706f0240561 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:13.855300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426f013-99ed-4766-bdda-d706f0240561 None admin] RBAC: Adding query filter params (role.id=8c039234db9640d0bdc0c2a629ff9e52, scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b, user.id=4eecd6829d5a4a319fa4af969eb3d39d) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:13.855300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426f013-99ed-4766-bdda-d706f0240561 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:13.856897 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426f013-99ed-4766-bdda-d706f0240561 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:13.864167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 98/198] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:13 2022] GET /identity/v3/role_assignments?user.id=4eecd6829d5a4a319fa4af969eb3d39d&scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b&role.id=8c039234db9640d0bdc0c2a629ff9e52 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:13.865072 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426f013-99ed-4766-bdda-d706f0240561 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:13.877895 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:13.890739 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:13.911861 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79d847-5c85-4466-a3ca-e3370d809376 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:13.966126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c3bd3-a43f-4cf0-ab61-e5556374902f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:13.966802 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c3bd3-a43f-4cf0-ab61-e5556374902f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:13.967086 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c3bd3-a43f-4cf0-ab61-e5556374902f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:13.967360 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c3bd3-a43f-4cf0-ab61-e5556374902f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:13.968214 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c3bd3-a43f-4cf0-ab61-e5556374902f None admin] RBAC: Adding query filter params (role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.project.id=57b2ea019ba3413bafb7714839f85a22, user.id=b8bc8c0a6f2640f69272813cdd67ca2e) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:13.968550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c3bd3-a43f-4cf0-ab61-e5556374902f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:13.970054 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c3bd3-a43f-4cf0-ab61-e5556374902f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:13.977198 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 101/199] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:13 2022] GET /identity/v3/role_assignments?user.id=b8bc8c0a6f2640f69272813cdd67ca2e&scope.project.id=57b2ea019ba3413bafb7714839f85a22&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:13.978341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c3bd3-a43f-4cf0-ab61-e5556374902f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:14.028888 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426f013-99ed-4766-bdda-d706f0240561 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:14.070859 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9506fcd-56e6-4d08-ad12-9f9a4a8c0f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:14.071547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9506fcd-56e6-4d08-ad12-9f9a4a8c0f77 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:14.071830 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9506fcd-56e6-4d08-ad12-9f9a4a8c0f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:14.072106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9506fcd-56e6-4d08-ad12-9f9a4a8c0f77 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:14.072934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9506fcd-56e6-4d08-ad12-9f9a4a8c0f77 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:14.073243 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9506fcd-56e6-4d08-ad12-9f9a4a8c0f77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:14.074677 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9506fcd-56e6-4d08-ad12-9f9a4a8c0f77 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:14.085871 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 99/200] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:14.091826 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9506fcd-56e6-4d08-ad12-9f9a4a8c0f77 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:14.113658 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c3bd3-a43f-4cf0-ab61-e5556374902f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:14.167669 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16995e-7adb-4ba7-a8e7-7129e53fbb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:14.168697 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16995e-7adb-4ba7-a8e7-7129e53fbb61 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:14.168987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16995e-7adb-4ba7-a8e7-7129e53fbb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:14.169267 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16995e-7adb-4ba7-a8e7-7129e53fbb61 None admin] PATH_INFO: `/v3/users/b8bc8c0a6f2640f69272813cdd67ca2e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:14.171954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16995e-7adb-4ba7-a8e7-7129e53fbb61 None admin] RBAC: Authorizing `identity:get_user(user_id=b8bc8c0a6f2640f69272813cdd67ca2e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:14.173592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16995e-7adb-4ba7-a8e7-7129e53fbb61 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:14.176245 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 102/201] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:13 2022] GET /identity/v3/users/b8bc8c0a6f2640f69272813cdd67ca2e => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:14.182234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16995e-7adb-4ba7-a8e7-7129e53fbb61 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:14.202790 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:14.220643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:14.255050 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9506fcd-56e6-4d08-ad12-9f9a4a8c0f77 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:14.304626 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4f145-3a33-4b0e-a120-6d332bc69347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:14.305535 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4f145-3a33-4b0e-a120-6d332bc69347 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:14.305827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4f145-3a33-4b0e-a120-6d332bc69347 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:14.306109 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4f145-3a33-4b0e-a120-6d332bc69347 None admin] PATH_INFO: `/v3/users/cc013382f7504c65a526dd7f68c9e78f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:14.307371 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4f145-3a33-4b0e-a120-6d332bc69347 None admin] RBAC: Authorizing `identity:get_user(user_id=cc013382f7504c65a526dd7f68c9e78f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:14.312537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4f145-3a33-4b0e-a120-6d332bc69347 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:14.312537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 100/202] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:14 2022] GET /identity/v3/users/cc013382f7504c65a526dd7f68c9e78f => generated 289 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:14.317078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4f145-3a33-4b0e-a120-6d332bc69347 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:14.336348 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16995e-7adb-4ba7-a8e7-7129e53fbb61 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:14.418249 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d09cc-6b77-470e-b548-08d3ff4ede61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:14.418915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d09cc-6b77-470e-b548-08d3ff4ede61 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:14.419202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d09cc-6b77-470e-b548-08d3ff4ede61 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:14.419483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d09cc-6b77-470e-b548-08d3ff4ede61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:14.497410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4f145-3a33-4b0e-a120-6d332bc69347 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:14.539079 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76ef39-76ae-4760-a5e2-6b10b7462278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:14.540183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76ef39-76ae-4760-a5e2-6b10b7462278 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:14.540476 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76ef39-76ae-4760-a5e2-6b10b7462278 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:14.540784 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76ef39-76ae-4760-a5e2-6b10b7462278 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:14.554004 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d09cc-6b77-470e-b548-08d3ff4ede61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:14.555573 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d09cc-6b77-470e-b548-08d3ff4ede61 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:14.560881 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:14.598460 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:14.634677 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76ef39-76ae-4760-a5e2-6b10b7462278 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:14.636859 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76ef39-76ae-4760-a5e2-6b10b7462278 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:14.717070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 103/203] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:14.722314 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d09cc-6b77-470e-b548-08d3ff4ede61 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:14.827929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 101/204] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:14.830614 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d09cc-6b77-470e-b548-08d3ff4ede61 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:14.834183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76ef39-76ae-4760-a5e2-6b10b7462278 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:14.872879 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:14.884899 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0aaf9-08e8-4e9e-883e-0dcfdc339602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:14.885777 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0aaf9-08e8-4e9e-883e-0dcfdc339602 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:14.886067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0aaf9-08e8-4e9e-883e-0dcfdc339602 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:14.886348 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0aaf9-08e8-4e9e-883e-0dcfdc339602 None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:14.889535 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0aaf9-08e8-4e9e-883e-0dcfdc339602 None admin] RBAC: Authorizing `identity:get_project(project_id=57b2ea019ba3413bafb7714839f85a22)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:14.891075 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0aaf9-08e8-4e9e-883e-0dcfdc339602 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:14.892922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 104/205] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:14 2022] GET /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:14.900749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0aaf9-08e8-4e9e-883e-0dcfdc339602 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:14.929706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:15.007720 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0aaf9-08e8-4e9e-883e-0dcfdc339602 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:15.043584 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728f2dd-8ffb-4b8c-8024-427b33711e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:15.044459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728f2dd-8ffb-4b8c-8024-427b33711e89 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:15.044927 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728f2dd-8ffb-4b8c-8024-427b33711e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:15.045352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728f2dd-8ffb-4b8c-8024-427b33711e89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:15.046496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728f2dd-8ffb-4b8c-8024-427b33711e89 None admin] RBAC: Adding query filter params (role.id=8c039234db9640d0bdc0c2a629ff9e52, scope.project.id=57b2ea019ba3413bafb7714839f85a22, user.id=b8bc8c0a6f2640f69272813cdd67ca2e) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:15.046988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728f2dd-8ffb-4b8c-8024-427b33711e89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:15.048603 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728f2dd-8ffb-4b8c-8024-427b33711e89 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:15.055599 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 105/206] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:14 2022] GET /identity/v3/role_assignments?user.id=b8bc8c0a6f2640f69272813cdd67ca2e&scope.project.id=57b2ea019ba3413bafb7714839f85a22&role.id=8c039234db9640d0bdc0c2a629ff9e52 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:15.089891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76ef39-76ae-4760-a5e2-6b10b7462278 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:15.150775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028050b7-770d-4b94-897e-e88e09703539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:15.151668 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028050b7-770d-4b94-897e-e88e09703539 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:15.151961 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028050b7-770d-4b94-897e-e88e09703539 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:15.152246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028050b7-770d-4b94-897e-e88e09703539 None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:15.153513 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028050b7-770d-4b94-897e-e88e09703539 None admin] RBAC: Authorizing `identity:get_project(project_id=09d730b36d8f4ab482dc9ed98757bb0b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:15.155124 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028050b7-770d-4b94-897e-e88e09703539 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:15.161323 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 102/207] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:14 2022] GET /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b => generated 298 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:15.167055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728f2dd-8ffb-4b8c-8024-427b33711e89 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:15.244424 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:15.316551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728f2dd-8ffb-4b8c-8024-427b33711e89 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:15.365744 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65182a-2d17-4286-a09e-70837ed85253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:15.366936 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65182a-2d17-4286-a09e-70837ed85253 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:15.367222 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65182a-2d17-4286-a09e-70837ed85253 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:15.367500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65182a-2d17-4286-a09e-70837ed85253 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:15.368347 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65182a-2d17-4286-a09e-70837ed85253 None admin] RBAC: Adding query filter params (role.id=e6c6a363f7034bd0a72b72ffe324aab8, scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b, user.id=cc013382f7504c65a526dd7f68c9e78f) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:15.368691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65182a-2d17-4286-a09e-70837ed85253 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:15.370162 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65182a-2d17-4286-a09e-70837ed85253 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:15.380716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 106/208] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:15 2022] GET /identity/v3/role_assignments?user.id=cc013382f7504c65a526dd7f68c9e78f&scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b&role.id=e6c6a363f7034bd0a72b72ffe324aab8 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:16.334040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b48f1cd-5951-4a96-827c-12f6e39669c2 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:16.334574 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b48f1cd-5951-4a96-827c-12f6e39669c2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:16.334652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b48f1cd-5951-4a96-827c-12f6e39669c2 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:16.336184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 103/209] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:16.341566 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb984e-f391-4a9d-97f8-1c75e6b12946 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:16.342377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb984e-f391-4a9d-97f8-1c75e6b12946 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:16.342864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb984e-f391-4a9d-97f8-1c75e6b12946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:16.361093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d6fb984e-f391-4a9d-97f8-1c75e6b12946 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:16.362610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d6fb984e-f391-4a9d-97f8-1c75e6b12946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:16.364625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbfc66-258f-4aef-bb82-519c44b34d65 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:16.364952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbfc66-258f-4aef-bb82-519c44b34d65 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:16.365346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbfc66-258f-4aef-bb82-519c44b34d65 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:16.367561 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 104/210] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:16.372496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927398a2-c631-4b3c-9a67-d0db0598dddb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:16.372829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927398a2-c631-4b3c-9a67-d0db0598dddb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:16.373113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927398a2-c631-4b3c-9a67-d0db0598dddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:16.391613 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-927398a2-c631-4b3c-9a67-d0db0598dddb None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:16.392810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-927398a2-c631-4b3c-9a67-d0db0598dddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:16.447279 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 107/211] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:16.469029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb984e-f391-4a9d-97f8-1c75e6b12946 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:16.470825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 105/212] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:16.494937 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927398a2-c631-4b3c-9a67-d0db0598dddb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:16.519453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:16.544322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:16.578310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb984e-f391-4a9d-97f8-1c75e6b12946 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:16.603959 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927398a2-c631-4b3c-9a67-d0db0598dddb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:16.611864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926b9ae-d394-4acd-b78c-d4eb7c4d6338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:16.612698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926b9ae-d394-4acd-b78c-d4eb7c4d6338 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:16.612982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926b9ae-d394-4acd-b78c-d4eb7c4d6338 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:16.613258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926b9ae-d394-4acd-b78c-d4eb7c4d6338 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:16.623758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926b9ae-d394-4acd-b78c-d4eb7c4d6338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:16.631332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e926b9ae-d394-4acd-b78c-d4eb7c4d6338 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:15:16.631332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:16.631332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:16.631332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:16.631332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:16.631332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:16.631332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:16.631332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:16.631332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:16.631332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:16.631332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:16.631332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:16.631332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:16.631332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:16.631332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:16.631332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:16.633036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:16.633036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:16.633036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:16.633036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:16.633036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:16.633036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:16.633036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:16.633036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:16.633036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:16.633036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:16.633036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:16.633036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:16.633036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:16.633036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:16.633036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:16.633036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:16.634127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:16.634127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:16.634127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:16.634127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:16.634127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:16.634127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:16.634127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:16.634127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:16.634127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:16.634127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:16.634127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:15:16.634127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:16.637881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 108/213] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:15:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:15:16.639012 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e841a60-0acc-43e6-a15d-07a23503af11 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:16.639303 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e841a60-0acc-43e6-a15d-07a23503af11 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:16.639582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e841a60-0acc-43e6-a15d-07a23503af11 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:16.640901 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 109/214] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:16.645010 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e841a60-0acc-43e6-a15d-07a23503af11 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:16.649627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848ea47-272e-4efc-b3c9-5a95e8d56c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:16.650750 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848ea47-272e-4efc-b3c9-5a95e8d56c93 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:16.651043 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848ea47-272e-4efc-b3c9-5a95e8d56c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:16.651323 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848ea47-272e-4efc-b3c9-5a95e8d56c93 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:16.662530 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848ea47-272e-4efc-b3c9-5a95e8d56c93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:16.671004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c848ea47-272e-4efc-b3c9-5a95e8d56c93 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:16.671004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:16.671004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:16.671004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:16.671004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:16.671004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:16.671004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:16.671004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:16.671004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:16.671004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:16.671004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:16.671004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:16.671004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:16.671004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:16.671004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:16.671004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:16.671879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:16.671879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:16.671879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:16.671879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:16.671879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:16.671879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:16.671879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:16.671879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:16.671879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:16.671879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:16.671879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:16.671879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:16.671879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:16.671879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:16.671879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:16.671879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:16.672687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:16.672687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:16.672687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:16.672687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:16.672687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:16.672687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:16.672687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:16.672687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:16.672687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:16.672687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:16.672687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:16.672687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:16.672687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 106/215] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:15:16.673793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d40cef-4b77-4a8b-9197-e1f2827270ec None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:16.673895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d40cef-4b77-4a8b-9197-e1f2827270ec None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:16.674181 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d40cef-4b77-4a8b-9197-e1f2827270ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:16.692040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-91d40cef-4b77-4a8b-9197-e1f2827270ec None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:16.693495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-91d40cef-4b77-4a8b-9197-e1f2827270ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:16.745722 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:16.749798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e841a60-0acc-43e6-a15d-07a23503af11 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:16.774306 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 107/216] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:16.774306 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d40cef-4b77-4a8b-9197-e1f2827270ec None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:16.793423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcae50-3422-4612-a51d-a7f1ef02c655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:16.794695 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcae50-3422-4612-a51d-a7f1ef02c655 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:16.795217 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcae50-3422-4612-a51d-a7f1ef02c655 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:16.796065 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcae50-3422-4612-a51d-a7f1ef02c655 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:16.797049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcae50-3422-4612-a51d-a7f1ef02c655 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:16.797531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcae50-3422-4612-a51d-a7f1ef02c655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:16.799633 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcae50-3422-4612-a51d-a7f1ef02c655 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:16.811936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:16.813361 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 110/217] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:16.813761 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcae50-3422-4612-a51d-a7f1ef02c655 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:16.887405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d40cef-4b77-4a8b-9197-e1f2827270ec None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:16.920497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcae50-3422-4612-a51d-a7f1ef02c655 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:16.923137 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27361c-21d3-4043-8ad8-022ac87651be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:16.923822 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27361c-21d3-4043-8ad8-022ac87651be None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:16.924199 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27361c-21d3-4043-8ad8-022ac87651be None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:16.924486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27361c-21d3-4043-8ad8-022ac87651be None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:16.925272 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b27361c-21d3-4043-8ad8-022ac87651be None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:16.925640 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b27361c-21d3-4043-8ad8-022ac87651be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:16.927131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b27361c-21d3-4043-8ad8-022ac87651be None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:16.940942 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 108/218] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:16.940942 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27361c-21d3-4043-8ad8-022ac87651be None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:16.959721 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf594f-2e70-42e5-8852-80c155c147a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:16.961112 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf594f-2e70-42e5-8852-80c155c147a3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:16.961391 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf594f-2e70-42e5-8852-80c155c147a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:16.961664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf594f-2e70-42e5-8852-80c155c147a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:16.962832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf594f-2e70-42e5-8852-80c155c147a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:16.964299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf594f-2e70-42e5-8852-80c155c147a3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:16.968547 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 111/219] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:16.968547 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf594f-2e70-42e5-8852-80c155c147a3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:17.000380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:17.054399 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:17.061121 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27361c-21d3-4043-8ad8-022ac87651be None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:17.081523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf594f-2e70-42e5-8852-80c155c147a3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:17.095971 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7052902-4805-4459-9c1f-df7dbca83f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:17.096869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7052902-4805-4459-9c1f-df7dbca83f42 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:17.097157 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7052902-4805-4459-9c1f-df7dbca83f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:17.097434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7052902-4805-4459-9c1f-df7dbca83f42 None admin] PATH_INFO: `/v3/users/b8bc8c0a6f2640f69272813cdd67ca2e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:17.098636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7052902-4805-4459-9c1f-df7dbca83f42 None admin] RBAC: Authorizing `identity:get_user(user_id=b8bc8c0a6f2640f69272813cdd67ca2e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:17.100086 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7052902-4805-4459-9c1f-df7dbca83f42 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:17.102302 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 109/220] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:16 2022] GET /identity/v3/users/b8bc8c0a6f2640f69272813cdd67ca2e => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:17.103060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7052902-4805-4459-9c1f-df7dbca83f42 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:17.122064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bdeec-e7a8-4a31-8431-637060703bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:17.122944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bdeec-e7a8-4a31-8431-637060703bde None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:17.123226 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bdeec-e7a8-4a31-8431-637060703bde None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:17.123654 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bdeec-e7a8-4a31-8431-637060703bde None admin] PATH_INFO: `/v3/users/d87b27ff26c54cbb843d087906fce4ba` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:17.138169 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41bdeec-e7a8-4a31-8431-637060703bde None admin] RBAC: Authorizing `identity:get_user(user_id=d87b27ff26c54cbb843d087906fce4ba)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:17.139669 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41bdeec-e7a8-4a31-8431-637060703bde None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:17.141557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 112/221] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:16 2022] GET /identity/v3/users/d87b27ff26c54cbb843d087906fce4ba => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:17.142146 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bdeec-e7a8-4a31-8431-637060703bde None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:17.223389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7052902-4805-4459-9c1f-df7dbca83f42 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:17.234948 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:17.247870 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bdeec-e7a8-4a31-8431-637060703bde None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:17.257458 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9dc9e-e9b6-4dd3-9e8a-b338b5e35181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:17.258064 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9dc9e-e9b6-4dd3-9e8a-b338b5e35181 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:17.258342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9dc9e-e9b6-4dd3-9e8a-b338b5e35181 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:17.258619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9dc9e-e9b6-4dd3-9e8a-b338b5e35181 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:17.259316 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9dc9e-e9b6-4dd3-9e8a-b338b5e35181 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:17.260789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9dc9e-e9b6-4dd3-9e8a-b338b5e35181 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:17.280615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 110/222] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 18 10:15:17 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:15:17.281619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9dc9e-e9b6-4dd3-9e8a-b338b5e35181 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:17.287413 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8f727-f461-4541-bb1f-609d2cb898b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:17.288606 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8f727-f461-4541-bb1f-609d2cb898b4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:17.288876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8f727-f461-4541-bb1f-609d2cb898b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:17.289359 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8f727-f461-4541-bb1f-609d2cb898b4 None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:17.290495 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8f727-f461-4541-bb1f-609d2cb898b4 None admin] RBAC: Authorizing `identity:get_project(project_id=57b2ea019ba3413bafb7714839f85a22)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:17.292055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8f727-f461-4541-bb1f-609d2cb898b4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:17.294151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 113/223] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:17 2022] GET /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:17.295213 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec292b0-7c81-4292-879d-f3d0a302e4d2 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:17.295506 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec292b0-7c81-4292-879d-f3d0a302e4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:17.295785 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec292b0-7c81-4292-879d-f3d0a302e4d2 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:17.301451 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 114/224] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:17.302272 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec292b0-7c81-4292-879d-f3d0a302e4d2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:17.313396 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:17.405817 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9dc9e-e9b6-4dd3-9e8a-b338b5e35181 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:17.433656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec292b0-7c81-4292-879d-f3d0a302e4d2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:17.445030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bd03f-5447-4043-a456-46f6c276c4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:17.445697 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bd03f-5447-4043-a456-46f6c276c4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:17.445984 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bd03f-5447-4043-a456-46f6c276c4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:17.446266 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bd03f-5447-4043-a456-46f6c276c4f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:17.471283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ef4fd-85ec-483b-b70c-c79c717b84d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:17.472174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ef4fd-85ec-483b-b70c-c79c717b84d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:17.472457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ef4fd-85ec-483b-b70c-c79c717b84d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:17.472758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ef4fd-85ec-483b-b70c-c79c717b84d2 None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22/users/b8bc8c0a6f2640f69272813cdd67ca2e/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:17.474730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ef4fd-85ec-483b-b70c-c79c717b84d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=57b2ea019ba3413bafb7714839f85a22, user_id=b8bc8c0a6f2640f69272813cdd67ca2e, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:17.476234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ef4fd-85ec-483b-b70c-c79c717b84d2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:17.490032 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 115/225] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:15:17 2022] PUT /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22/users/b8bc8c0a6f2640f69272813cdd67ca2e/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:15:17.492123 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f894bbc-4533-46f9-88f0-dcc2aa90e15d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:17.492410 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f894bbc-4533-46f9-88f0-dcc2aa90e15d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:17.492708 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f894bbc-4533-46f9-88f0-dcc2aa90e15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:17.516789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6f894bbc-4533-46f9-88f0-dcc2aa90e15d None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:17.518002 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6f894bbc-4533-46f9-88f0-dcc2aa90e15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:17.539310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bd03f-5447-4043-a456-46f6c276c4f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:17.540942 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bd03f-5447-4043-a456-46f6c276c4f1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:17.546742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:17.574279 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:17.627130 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 116/226] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:17.654136 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f894bbc-4533-46f9-88f0-dcc2aa90e15d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:17.688538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 111/227] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:17.691948 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bd03f-5447-4043-a456-46f6c276c4f1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:17.796591 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f894bbc-4533-46f9-88f0-dcc2aa90e15d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:17.814629 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bd03f-5447-4043-a456-46f6c276c4f1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:17.837754 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74e617-51a0-4ad4-b46c-5aad29189170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:17.838711 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74e617-51a0-4ad4-b46c-5aad29189170 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:17.840844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74e617-51a0-4ad4-b46c-5aad29189170 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:17.841304 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74e617-51a0-4ad4-b46c-5aad29189170 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:17.841934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:17.856305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74e617-51a0-4ad4-b46c-5aad29189170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:17.859645 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e526a-7714-4db1-82fa-806b40dfc3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:17.860681 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e526a-7714-4db1-82fa-806b40dfc3bf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:17.861061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e526a-7714-4db1-82fa-806b40dfc3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:17.861428 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e526a-7714-4db1-82fa-806b40dfc3bf None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:17.862729 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e526a-7714-4db1-82fa-806b40dfc3bf None admin] RBAC: Authorizing `identity:get_project(project_id=57b2ea019ba3413bafb7714839f85a22)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:17.864266 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e526a-7714-4db1-82fa-806b40dfc3bf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:17.866328 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 112/228] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:17 2022] GET /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:17.867458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-6c74e617-51a0-4ad4-b46c-5aad29189170 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:17.867458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:17.867458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:17.867458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:17.867458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:17.867458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:17.867458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:17.867458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:17.867458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:17.867458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:17.867458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:17.867458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:17.867458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:17.867458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:17.867458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:17.867458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:17.868310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:17.868310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:17.868310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:17.868310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:17.868310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:17.868310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:17.868310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:17.868310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:17.868310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:17.868310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:17.868310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:17.868310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:17.868310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:17.868310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:17.868310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:17.868310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:17.869154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:17.869154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:17.869154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:17.869154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:17.869154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:17.869154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:17.869154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:17.869154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:17.869154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:17.869154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:17.869154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:17.869154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:17.869752 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 117/229] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:15:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:17.871890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e526a-7714-4db1-82fa-806b40dfc3bf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:17.882312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74e617-51a0-4ad4-b46c-5aad29189170 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:17.893842 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:18.038823 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e526a-7714-4db1-82fa-806b40dfc3bf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:18.043847 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74e617-51a0-4ad4-b46c-5aad29189170 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:18.104649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa58714-e4c0-4151-a285-e882d1df30d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:18.105634 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa58714-e4c0-4151-a285-e882d1df30d7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:18.106131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa58714-e4c0-4151-a285-e882d1df30d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:18.106613 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa58714-e4c0-4151-a285-e882d1df30d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:18.107667 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa58714-e4c0-4151-a285-e882d1df30d7 None admin] RBAC: Adding query filter params (role.id=2820090979cb487db491f835e03e6471, scope.project.id=57b2ea019ba3413bafb7714839f85a22, user.id=d87b27ff26c54cbb843d087906fce4ba) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:18.108185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa58714-e4c0-4151-a285-e882d1df30d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:18.109892 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa58714-e4c0-4151-a285-e882d1df30d7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:18.114026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c981346-491a-4b14-9b27-b215e72abb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:18.115033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c981346-491a-4b14-9b27-b215e72abb3b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:18.115318 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c981346-491a-4b14-9b27-b215e72abb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:18.115597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c981346-491a-4b14-9b27-b215e72abb3b None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:18.116952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c981346-491a-4b14-9b27-b215e72abb3b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:18.117261 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c981346-491a-4b14-9b27-b215e72abb3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:18.118267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 113/230] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:17 2022] GET /identity/v3/role_assignments?user.id=d87b27ff26c54cbb843d087906fce4ba&scope.project.id=57b2ea019ba3413bafb7714839f85a22&role.id=2820090979cb487db491f835e03e6471 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:18.121115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c981346-491a-4b14-9b27-b215e72abb3b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:18.131640 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 118/231] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:15:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:18.140567 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa58714-e4c0-4151-a285-e882d1df30d7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:18.212986 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:18.273275 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa58714-e4c0-4151-a285-e882d1df30d7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:18.325454 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede456c-0232-4b8b-a79a-56f594a5edc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:18.326349 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede456c-0232-4b8b-a79a-56f594a5edc5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:18.326634 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede456c-0232-4b8b-a79a-56f594a5edc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:18.326913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede456c-0232-4b8b-a79a-56f594a5edc5 None admin] PATH_INFO: `/v3/users/b8bc8c0a6f2640f69272813cdd67ca2e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:18.333405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede456c-0232-4b8b-a79a-56f594a5edc5 None admin] RBAC: Authorizing `identity:get_user(user_id=b8bc8c0a6f2640f69272813cdd67ca2e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:18.335466 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede456c-0232-4b8b-a79a-56f594a5edc5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:18.337619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 114/232] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:18 2022] GET /identity/v3/users/b8bc8c0a6f2640f69272813cdd67ca2e => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:18.343380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c981346-491a-4b14-9b27-b215e72abb3b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:18.446045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:18.487948 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c981346-491a-4b14-9b27-b215e72abb3b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:18.527915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c3d86-131b-4c30-8353-3041fd5ea230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:18.528811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c3d86-131b-4c30-8353-3041fd5ea230 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:18.529095 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c3d86-131b-4c30-8353-3041fd5ea230 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:18.529367 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c3d86-131b-4c30-8353-3041fd5ea230 None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:18.530569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c3d86-131b-4c30-8353-3041fd5ea230 None admin] RBAC: Authorizing `identity:get_project(project_id=57b2ea019ba3413bafb7714839f85a22)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:18.532034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c3d86-131b-4c30-8353-3041fd5ea230 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:18.533763 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 119/233] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:18 2022] GET /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:18.539525 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede456c-0232-4b8b-a79a-56f594a5edc5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:18.688349 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede456c-0232-4b8b-a79a-56f594a5edc5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:18.708949 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:18.724073 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c2898-9bbc-4f04-b2d5-31c1c7e90f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:18.724989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c2898-9bbc-4f04-b2d5-31c1c7e90f24 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:18.725274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c2898-9bbc-4f04-b2d5-31c1c7e90f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:18.726123 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c2898-9bbc-4f04-b2d5-31c1c7e90f24 None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22/users/b8bc8c0a6f2640f69272813cdd67ca2e/roles/8c039234db9640d0bdc0c2a629ff9e52` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:18.729488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c2898-9bbc-4f04-b2d5-31c1c7e90f24 None admin] RBAC: Authorizing `identity:create_grant(project_id=57b2ea019ba3413bafb7714839f85a22, user_id=b8bc8c0a6f2640f69272813cdd67ca2e, role_id=8c039234db9640d0bdc0c2a629ff9e52)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:18.731119 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c2898-9bbc-4f04-b2d5-31c1c7e90f24 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:18.746256 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 115/234] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:15:18 2022] PUT /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22/users/b8bc8c0a6f2640f69272813cdd67ca2e/roles/8c039234db9640d0bdc0c2a629ff9e52 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:15:19.905254 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41753bed-470b-465a-8b4a-35eb2e29bbe9 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:19.905775 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41753bed-470b-465a-8b4a-35eb2e29bbe9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:19.905858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41753bed-470b-465a-8b4a-35eb2e29bbe9 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:19.907433 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 120/235] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:19.912725 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16b2b1-9bff-40d6-b9b4-b5f42e1e8bb1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:19.913688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16b2b1-9bff-40d6-b9b4-b5f42e1e8bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:19.913830 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16b2b1-9bff-40d6-b9b4-b5f42e1e8bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:19.941506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2b16b2b1-9bff-40d6-b9b4-b5f42e1e8bb1 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:19.942705 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2b16b2b1-9bff-40d6-b9b4-b5f42e1e8bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:20.013296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337edd7-a19e-431b-8e0d-27c44105718f None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:20.013637 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337edd7-a19e-431b-8e0d-27c44105718f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:20.013858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337edd7-a19e-431b-8e0d-27c44105718f None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:20.015236 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 121/236] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:15:20.023656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed667b51-ebbc-4c6d-864b-82ddd51aee48 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:20.023939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed667b51-ebbc-4c6d-864b-82ddd51aee48 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:20.024215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed667b51-ebbc-4c6d-864b-82ddd51aee48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:20.045925 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ed667b51-ebbc-4c6d-864b-82ddd51aee48 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:20.047119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ed667b51-ebbc-4c6d-864b-82ddd51aee48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:20.066212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 116/237] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:20.083498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009507c-07d2-4f03-ac9a-8ced6d456ee6 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:20.083785 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009507c-07d2-4f03-ac9a-8ced6d456ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:20.084062 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009507c-07d2-4f03-ac9a-8ced6d456ee6 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:20.086052 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 117/238] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:15:20.090889 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c49ff-1e24-4958-adc9-ff7e275625b5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:20.091174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c49ff-1e24-4958-adc9-ff7e275625b5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:20.091450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c49ff-1e24-4958-adc9-ff7e275625b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:20.113542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b79c49ff-1e24-4958-adc9-ff7e275625b5 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:20.114723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b79c49ff-1e24-4958-adc9-ff7e275625b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:20.168991 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 122/239] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:20.169918 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed667b51-ebbc-4c6d-864b-82ddd51aee48 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:20.210570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 118/240] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:20.211438 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c49ff-1e24-4958-adc9-ff7e275625b5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:20.258061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:20.272254 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:20.314373 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed667b51-ebbc-4c6d-864b-82ddd51aee48 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:20.329904 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c49ff-1e24-4958-adc9-ff7e275625b5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:20.351995 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53a21b-c3dd-45cd-8581-14fcebc83a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:20.353023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53a21b-c3dd-45cd-8581-14fcebc83a95 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:20.353706 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53a21b-c3dd-45cd-8581-14fcebc83a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:20.354076 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53a21b-c3dd-45cd-8581-14fcebc83a95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:20.366903 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53a21b-c3dd-45cd-8581-14fcebc83a95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:20.370808 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094ec43-614d-43f6-b3dd-56583e31ab6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:20.371786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094ec43-614d-43f6-b3dd-56583e31ab6b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:20.372218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094ec43-614d-43f6-b3dd-56583e31ab6b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:20.372667 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094ec43-614d-43f6-b3dd-56583e31ab6b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:20.375753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-bc53a21b-c3dd-45cd-8581-14fcebc83a95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:15:20.375753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:20.375753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:20.375753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:20.375753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:20.375753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:20.375753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:20.375753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:20.375753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:20.375753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:20.375753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:20.375753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:20.375753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:20.375753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:20.375753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:20.375753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:20.376764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:20.376764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:20.376764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:20.376764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:20.376764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:20.376764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:20.376764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:20.376764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:20.376764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:20.376764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:20.376764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:20.376764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:20.376764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:20.376764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:20.376764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:20.376764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:20.377568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:20.377568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:20.377568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:20.377568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:20.377568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:20.377568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:20.377568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:20.377568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:20.377568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:20.377568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:20.377568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:15:20.377568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:20.382705 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 123/241] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:15:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:20.383617 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53a21b-c3dd-45cd-8581-14fcebc83a95 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:20.388645 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094ec43-614d-43f6-b3dd-56583e31ab6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:20.397406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-5094ec43-614d-43f6-b3dd-56583e31ab6b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:15:20.397406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:20.397406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:20.397406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:20.397406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:20.397406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:20.397406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:20.397406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:20.397406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:20.397406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:20.397406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:20.397406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:20.397406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:20.397406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:20.397406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:20.397406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:20.398293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:20.398293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:20.398293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:20.398293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:20.398293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:20.398293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:20.398293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:20.398293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:20.398293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:20.398293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:20.398293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:20.398293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:20.398293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:20.398293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:20.398293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:20.398293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:20.399101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:20.399101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:20.399101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:20.399101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:20.399101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:20.399101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:20.399101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:20.399101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:20.399101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:20.399101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:20.399101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:15:20.399101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:20.401584 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 119/242] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:20.402595 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094ec43-614d-43f6-b3dd-56583e31ab6b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:20.497922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53a21b-c3dd-45cd-8581-14fcebc83a95 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:20.508868 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:20.512125 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094ec43-614d-43f6-b3dd-56583e31ab6b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:20.527602 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:20.537821 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3d4bf-e8cd-4720-b6bf-87fd1911fd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:20.538746 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3d4bf-e8cd-4720-b6bf-87fd1911fd99 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:20.539032 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3d4bf-e8cd-4720-b6bf-87fd1911fd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:20.539305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3d4bf-e8cd-4720-b6bf-87fd1911fd99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:20.540621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3d4bf-e8cd-4720-b6bf-87fd1911fd99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:20.542083 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3d4bf-e8cd-4720-b6bf-87fd1911fd99 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:20.544161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 124/243] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:20.549362 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3d4bf-e8cd-4720-b6bf-87fd1911fd99 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:20.552431 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ede37c-e759-48b5-b60b-1eeb8b0b9cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:20.554191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ede37c-e759-48b5-b60b-1eeb8b0b9cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:20.554631 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ede37c-e759-48b5-b60b-1eeb8b0b9cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:20.555077 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ede37c-e759-48b5-b60b-1eeb8b0b9cf8 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:20.555993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ede37c-e759-48b5-b60b-1eeb8b0b9cf8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:20.556451 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ede37c-e759-48b5-b60b-1eeb8b0b9cf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:20.558045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ede37c-e759-48b5-b60b-1eeb8b0b9cf8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:20.567334 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 120/244] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:20.568290 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ede37c-e759-48b5-b60b-1eeb8b0b9cf8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:20.664290 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3d4bf-e8cd-4720-b6bf-87fd1911fd99 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:20.676616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ede37c-e759-48b5-b60b-1eeb8b0b9cf8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:20.701101 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404c24f-7349-4dec-b447-6102c95e0b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:20.701773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404c24f-7349-4dec-b447-6102c95e0b51 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:20.702105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404c24f-7349-4dec-b447-6102c95e0b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:20.702383 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404c24f-7349-4dec-b447-6102c95e0b51 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:20.703248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404c24f-7349-4dec-b447-6102c95e0b51 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:20.703557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404c24f-7349-4dec-b447-6102c95e0b51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:20.705084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404c24f-7349-4dec-b447-6102c95e0b51 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:20.715728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02554f-8100-453f-8563-25f8c5536d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:20.716639 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02554f-8100-453f-8563-25f8c5536d0b None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:20.717076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02554f-8100-453f-8563-25f8c5536d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:20.717527 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02554f-8100-453f-8563-25f8c5536d0b None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:20.718458 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02554f-8100-453f-8563-25f8c5536d0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:20.718574 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 125/245] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:20.719482 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404c24f-7349-4dec-b447-6102c95e0b51 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:20.720164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02554f-8100-453f-8563-25f8c5536d0b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:20.741972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 121/246] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 18 10:15:20 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:15:20.743477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510236f8-1493-40cf-993a-49c7e7cb31ad None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:20.743772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510236f8-1493-40cf-993a-49c7e7cb31ad None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:20.744056 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510236f8-1493-40cf-993a-49c7e7cb31ad None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:20.750192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 122/247] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:20.750192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510236f8-1493-40cf-993a-49c7e7cb31ad None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:20.770979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:20.806235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:20.869292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404c24f-7349-4dec-b447-6102c95e0b51 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:20.909590 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02b1b9-1e05-4e47-b1d7-2a2dbacc224f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:20.910484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510236f8-1493-40cf-993a-49c7e7cb31ad None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:20.911582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02b1b9-1e05-4e47-b1d7-2a2dbacc224f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:20.912099 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02b1b9-1e05-4e47-b1d7-2a2dbacc224f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:20.912611 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02b1b9-1e05-4e47-b1d7-2a2dbacc224f None admin] PATH_INFO: `/v3/users/b8bc8c0a6f2640f69272813cdd67ca2e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:20.915390 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e02b1b9-1e05-4e47-b1d7-2a2dbacc224f None admin] RBAC: Authorizing `identity:get_user(user_id=b8bc8c0a6f2640f69272813cdd67ca2e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:20.920238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e02b1b9-1e05-4e47-b1d7-2a2dbacc224f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:20.920238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 126/248] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:20 2022] GET /identity/v3/users/b8bc8c0a6f2640f69272813cdd67ca2e => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:20.924108 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff0d97-1bf3-4264-b58f-64f317ddaf8f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:20.924398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff0d97-1bf3-4264-b58f-64f317ddaf8f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:20.924699 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff0d97-1bf3-4264-b58f-64f317ddaf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:20.945206 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-68ff0d97-1bf3-4264-b58f-64f317ddaf8f None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:20.946400 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-68ff0d97-1bf3-4264-b58f-64f317ddaf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:20.956222 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d61dd-a23a-45e5-97e0-0197fc6a7b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:20.957140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d61dd-a23a-45e5-97e0-0197fc6a7b03 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:20.957425 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d61dd-a23a-45e5-97e0-0197fc6a7b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:20.957710 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d61dd-a23a-45e5-97e0-0197fc6a7b03 None admin] PATH_INFO: `/v3/users/8a57c25e3f0141079354e411060bfd18` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:20.974577 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d61dd-a23a-45e5-97e0-0197fc6a7b03 None admin] RBAC: Authorizing `identity:get_user(user_id=8a57c25e3f0141079354e411060bfd18)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:20.976719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d61dd-a23a-45e5-97e0-0197fc6a7b03 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:20.978737 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 123/249] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:20 2022] GET /identity/v3/users/8a57c25e3f0141079354e411060bfd18 => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:20.979554 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d61dd-a23a-45e5-97e0-0197fc6a7b03 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:21.031435 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 127/250] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:21.032036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff0d97-1bf3-4264-b58f-64f317ddaf8f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:21.076391 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:21.100000 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:21.106024 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d61dd-a23a-45e5-97e0-0197fc6a7b03 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:21.149490 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff0d97-1bf3-4264-b58f-64f317ddaf8f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:21.153020 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b1e3a-27fe-47d1-a8e9-dbff6a9d3ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:21.155364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b1e3a-27fe-47d1-a8e9-dbff6a9d3ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:21.155364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b1e3a-27fe-47d1-a8e9-dbff6a9d3ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:21.155364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b1e3a-27fe-47d1-a8e9-dbff6a9d3ac8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:21.203180 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e8535-4a30-44f3-8848-4e43cad65fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:21.203840 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e8535-4a30-44f3-8848-4e43cad65fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:21.204117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e8535-4a30-44f3-8848-4e43cad65fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:21.204389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e8535-4a30-44f3-8848-4e43cad65fc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:21.241493 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b1e3a-27fe-47d1-a8e9-dbff6a9d3ac8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:21.242975 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b1e3a-27fe-47d1-a8e9-dbff6a9d3ac8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:21.291465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e8535-4a30-44f3-8848-4e43cad65fc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:21.294287 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e8535-4a30-44f3-8848-4e43cad65fc4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:21.325017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:21.351702 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 124/251] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:21.352850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b1e3a-27fe-47d1-a8e9-dbff6a9d3ac8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:21.369334 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:21.432593 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 128/252] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:21.434326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff567d7-1c1b-4eb7-a534-ef3170a13f73 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:21.435251 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff567d7-1c1b-4eb7-a534-ef3170a13f73 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:21.435616 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff567d7-1c1b-4eb7-a534-ef3170a13f73 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:21.437562 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 129/253] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:21.438619 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff567d7-1c1b-4eb7-a534-ef3170a13f73 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:21.471379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b1e3a-27fe-47d1-a8e9-dbff6a9d3ac8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:21.512276 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402838d-d3df-4f84-ae00-ec0e779be801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:21.513306 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402838d-d3df-4f84-ae00-ec0e779be801 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:21.513660 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402838d-d3df-4f84-ae00-ec0e779be801 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:21.514009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402838d-d3df-4f84-ae00-ec0e779be801 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:21.528874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a402838d-d3df-4f84-ae00-ec0e779be801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:21.538706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a402838d-d3df-4f84-ae00-ec0e779be801 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:21.538706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:21.538706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:21.538706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:21.538706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:21.538706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:21.538706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:21.538706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:21.538706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:21.538706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:21.538706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:21.538706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:21.538706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:21.538706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:21.538706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:21.538706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:21.539644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:21.539644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:21.539644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:21.539644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:21.539644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:21.539644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:21.539644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:21.539644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:21.539644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:21.539644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:21.539644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:21.539644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:21.539644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:21.539644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:21.539644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:21.539644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:21.540450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:21.540450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:21.540450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:21.540450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:21.540450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:21.540450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:21.540450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:21.540450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:21.540450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:21.540450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:21.540450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:21.540450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:21.545021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 125/254] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:21.546600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402838d-d3df-4f84-ae00-ec0e779be801 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:21.573987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff567d7-1c1b-4eb7-a534-ef3170a13f73 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:21.598503 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:21.617909 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9e67c-5032-4487-b7b9-72d8afc97e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:21.619832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9e67c-5032-4487-b7b9-72d8afc97e5e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:21.620355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9e67c-5032-4487-b7b9-72d8afc97e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:21.621101 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9e67c-5032-4487-b7b9-72d8afc97e5e None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:21.623217 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9e67c-5032-4487-b7b9-72d8afc97e5e None admin] RBAC: Authorizing `identity:get_project(project_id=57b2ea019ba3413bafb7714839f85a22)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:21.625987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9e67c-5032-4487-b7b9-72d8afc97e5e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:21.627924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 130/255] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:21 2022] GET /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:21.630297 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a0db3-3b98-4b4c-839c-1cb0d524dc63 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:21.630815 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a0db3-3b98-4b4c-839c-1cb0d524dc63 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:21.631540 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a0db3-3b98-4b4c-839c-1cb0d524dc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:21.654152 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-915a0db3-3b98-4b4c-839c-1cb0d524dc63 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:21.656177 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-915a0db3-3b98-4b4c-839c-1cb0d524dc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:21.666625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402838d-d3df-4f84-ae00-ec0e779be801 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:21.678878 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:21.703614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01feebda-b3bf-4d23-bca3-3cc413cbeb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:21.704592 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01feebda-b3bf-4d23-bca3-3cc413cbeb47 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:21.704842 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01feebda-b3bf-4d23-bca3-3cc413cbeb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:21.705124 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01feebda-b3bf-4d23-bca3-3cc413cbeb47 None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:21.707332 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01feebda-b3bf-4d23-bca3-3cc413cbeb47 None admin] RBAC: Authorizing `identity:get_project(project_id=57b2ea019ba3413bafb7714839f85a22)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:21.708167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01feebda-b3bf-4d23-bca3-3cc413cbeb47 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:21.711038 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 126/256] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:21 2022] GET /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:21.711922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01feebda-b3bf-4d23-bca3-3cc413cbeb47 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:21.751772 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 131/257] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:21.753009 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a0db3-3b98-4b4c-839c-1cb0d524dc63 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:21.833235 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01feebda-b3bf-4d23-bca3-3cc413cbeb47 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:21.837064 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:21.870696 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a0db3-3b98-4b4c-839c-1cb0d524dc63 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:21.883266 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab1771-924c-4136-bb33-a438078121e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:21.883940 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab1771-924c-4136-bb33-a438078121e5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:21.884227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab1771-924c-4136-bb33-a438078121e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:21.884624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab1771-924c-4136-bb33-a438078121e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:21.885388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab1771-924c-4136-bb33-a438078121e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:21.885711 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab1771-924c-4136-bb33-a438078121e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:21.887111 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab1771-924c-4136-bb33-a438078121e5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:21.900334 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 127/258] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:21.901232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab1771-924c-4136-bb33-a438078121e5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:21.912804 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bca6d-447c-4557-ad3c-b6f7749f4461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:21.914504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bca6d-447c-4557-ad3c-b6f7749f4461 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:21.915352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bca6d-447c-4557-ad3c-b6f7749f4461 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:21.915783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bca6d-447c-4557-ad3c-b6f7749f4461 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:21.917087 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bca6d-447c-4557-ad3c-b6f7749f4461 None admin] RBAC: Adding query filter params (role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.project.id=57b2ea019ba3413bafb7714839f85a22, user.id=b8bc8c0a6f2640f69272813cdd67ca2e) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:21.917550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bca6d-447c-4557-ad3c-b6f7749f4461 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:21.919908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bca6d-447c-4557-ad3c-b6f7749f4461 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:21.928001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 132/259] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:21 2022] GET /identity/v3/role_assignments?user.id=b8bc8c0a6f2640f69272813cdd67ca2e&scope.project.id=57b2ea019ba3413bafb7714839f85a22&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:21.929847 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bca6d-447c-4557-ad3c-b6f7749f4461 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:21.952048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:22.051704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bca6d-447c-4557-ad3c-b6f7749f4461 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:22.105383 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0bd73-124f-48e8-b090-9ef75ee044ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:22.106546 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0bd73-124f-48e8-b090-9ef75ee044ac None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:22.107032 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0bd73-124f-48e8-b090-9ef75ee044ac None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:22.107507 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0bd73-124f-48e8-b090-9ef75ee044ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:22.121335 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0bd73-124f-48e8-b090-9ef75ee044ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:22.130440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-93c0bd73-124f-48e8-b090-9ef75ee044ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:22.130440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:22.130440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:22.130440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:22.130440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:22.130440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:22.130440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:22.130440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:22.130440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:22.130440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:22.130440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:22.130440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:22.130440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:22.130440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:22.130440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:22.130440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:22.131320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:22.131320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:22.131320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:22.131320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:22.131320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:22.131320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:22.131320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:22.131320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:22.131320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:22.131320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:22.131320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:22.131320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:22.131320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:22.131320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:22.131320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:22.131320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:22.132737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:22.132737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:22.132737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:22.132737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:22.132737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:22.132737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:22.132737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:22.132737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:22.132737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:22.132737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:22.132737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:22.132737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:22.134101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab1771-924c-4136-bb33-a438078121e5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:22.135735 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 133/260] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:15:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:22.137412 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0bd73-124f-48e8-b090-9ef75ee044ac None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:22.182921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90fdb13-a977-4cb9-adbc-ad63b743908e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:22.183607 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fdb13-a977-4cb9-adbc-ad63b743908e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:22.183894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fdb13-a977-4cb9-adbc-ad63b743908e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:22.184171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fdb13-a977-4cb9-adbc-ad63b743908e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:22.185062 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fdb13-a977-4cb9-adbc-ad63b743908e None admin] RBAC: Adding query filter params (role.id=e6c6a363f7034bd0a72b72ffe324aab8, scope.project.id=57b2ea019ba3413bafb7714839f85a22, user.id=8a57c25e3f0141079354e411060bfd18) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:22.185376 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fdb13-a977-4cb9-adbc-ad63b743908e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:22.186823 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fdb13-a977-4cb9-adbc-ad63b743908e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:22.193551 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 128/261] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:21 2022] GET /identity/v3/role_assignments?user.id=8a57c25e3f0141079354e411060bfd18&scope.project.id=57b2ea019ba3413bafb7714839f85a22&role.id=e6c6a363f7034bd0a72b72ffe324aab8 => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:22.194899 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90fdb13-a977-4cb9-adbc-ad63b743908e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:22.232550 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:22.243194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:22.314842 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0bd73-124f-48e8-b090-9ef75ee044ac None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:22.364081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90fdb13-a977-4cb9-adbc-ad63b743908e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:22.392319 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d97f9f-f15b-4fb8-a3bb-16484cd2bc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:22.393245 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d97f9f-f15b-4fb8-a3bb-16484cd2bc10 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:22.393531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d97f9f-f15b-4fb8-a3bb-16484cd2bc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:22.393806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d97f9f-f15b-4fb8-a3bb-16484cd2bc10 None admin] PATH_INFO: `/v3/users/d87b27ff26c54cbb843d087906fce4ba` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:22.395168 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d97f9f-f15b-4fb8-a3bb-16484cd2bc10 None admin] RBAC: Authorizing `identity:get_user(user_id=d87b27ff26c54cbb843d087906fce4ba)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:22.396697 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d97f9f-f15b-4fb8-a3bb-16484cd2bc10 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:22.399013 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 134/262] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:22 2022] GET /identity/v3/users/d87b27ff26c54cbb843d087906fce4ba => generated 297 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:22.405946 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d97f9f-f15b-4fb8-a3bb-16484cd2bc10 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:22.434691 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7906813-e134-4081-859e-92ca05690dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:22.435418 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7906813-e134-4081-859e-92ca05690dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:22.435707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7906813-e134-4081-859e-92ca05690dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:22.435984 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7906813-e134-4081-859e-92ca05690dd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:22.436767 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7906813-e134-4081-859e-92ca05690dd9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:22.437308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7906813-e134-4081-859e-92ca05690dd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:22.438700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7906813-e134-4081-859e-92ca05690dd9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:22.456752 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 129/263] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:15:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:22.464481 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7906813-e134-4081-859e-92ca05690dd9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:22.578571 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d97f9f-f15b-4fb8-a3bb-16484cd2bc10 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:22.642668 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:22.648443 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba2c65-5f84-4abf-8508-0f557f21e93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:22.649345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba2c65-5f84-4abf-8508-0f557f21e93a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:22.649634 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba2c65-5f84-4abf-8508-0f557f21e93a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:22.649912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba2c65-5f84-4abf-8508-0f557f21e93a None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:22.651236 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba2c65-5f84-4abf-8508-0f557f21e93a None admin] RBAC: Authorizing `identity:get_project(project_id=57b2ea019ba3413bafb7714839f85a22)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:22.652737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba2c65-5f84-4abf-8508-0f557f21e93a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:22.656812 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 135/264] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:22 2022] GET /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:22.659590 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7906813-e134-4081-859e-92ca05690dd9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:22.661876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba2c65-5f84-4abf-8508-0f557f21e93a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:22.691579 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:22.725665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b6dd0-6848-4d1b-9376-0e246890b1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:22.726936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b6dd0-6848-4d1b-9376-0e246890b1bc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:22.727419 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b6dd0-6848-4d1b-9376-0e246890b1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:22.727890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b6dd0-6848-4d1b-9376-0e246890b1bc None admin] PATH_INFO: `/v3/users/b8bc8c0a6f2640f69272813cdd67ca2e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:22.729614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b6dd0-6848-4d1b-9376-0e246890b1bc None admin] RBAC: Authorizing `identity:get_user(user_id=b8bc8c0a6f2640f69272813cdd67ca2e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:22.731271 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b6dd0-6848-4d1b-9376-0e246890b1bc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:22.742627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 130/265] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:22 2022] GET /identity/v3/users/b8bc8c0a6f2640f69272813cdd67ca2e => generated 283 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:22.747951 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b6dd0-6848-4d1b-9376-0e246890b1bc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:22.786356 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba2c65-5f84-4abf-8508-0f557f21e93a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:22.827038 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185629f0-9ed8-4ec8-bbd7-b031aaa6fb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:22.828343 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185629f0-9ed8-4ec8-bbd7-b031aaa6fb83 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:22.828854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185629f0-9ed8-4ec8-bbd7-b031aaa6fb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:22.829328 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185629f0-9ed8-4ec8-bbd7-b031aaa6fb83 None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22/users/d87b27ff26c54cbb843d087906fce4ba/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:22.832011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185629f0-9ed8-4ec8-bbd7-b031aaa6fb83 None admin] RBAC: Authorizing `identity:create_grant(project_id=57b2ea019ba3413bafb7714839f85a22, user_id=d87b27ff26c54cbb843d087906fce4ba, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:22.834303 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185629f0-9ed8-4ec8-bbd7-b031aaa6fb83 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:22.858566 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 136/266] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:15:22 2022] PUT /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22/users/d87b27ff26c54cbb843d087906fce4ba/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:15:22.910234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b6dd0-6848-4d1b-9376-0e246890b1bc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:22.950172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2379c61-845a-472e-a6a6-915d6e98c154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:22.951222 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2379c61-845a-472e-a6a6-915d6e98c154 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:22.951739 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2379c61-845a-472e-a6a6-915d6e98c154 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:22.952223 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2379c61-845a-472e-a6a6-915d6e98c154 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:22.998218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:23.052999 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2379c61-845a-472e-a6a6-915d6e98c154 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:23.055442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2379c61-845a-472e-a6a6-915d6e98c154 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:23.159893 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 131/267] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 10:15:23.166155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185629f0-9ed8-4ec8-bbd7-b031aaa6fb83 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:23.269401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:23.308178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97490a3-13f1-43bb-94ed-8ec85de02e32 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:23.308479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97490a3-13f1-43bb-94ed-8ec85de02e32 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:23.308772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97490a3-13f1-43bb-94ed-8ec85de02e32 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:23.310013 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185629f0-9ed8-4ec8-bbd7-b031aaa6fb83 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:23.310417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 132/268] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:23.325868 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04113797-1f18-4382-bd2e-9d4fb716c6ad None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:23.326151 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04113797-1f18-4382-bd2e-9d4fb716c6ad None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:23.326439 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04113797-1f18-4382-bd2e-9d4fb716c6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:23.355602 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32b926-9b56-4747-ba1d-6f874ba604c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:23.356474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32b926-9b56-4747-ba1d-6f874ba604c0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:23.356779 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32b926-9b56-4747-ba1d-6f874ba604c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:23.357054 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32b926-9b56-4747-ba1d-6f874ba604c0 None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:23.358507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-04113797-1f18-4382-bd2e-9d4fb716c6ad None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:23.359709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-04113797-1f18-4382-bd2e-9d4fb716c6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:23.362211 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32b926-9b56-4747-ba1d-6f874ba604c0 None admin] RBAC: Authorizing `identity:get_project(project_id=57b2ea019ba3413bafb7714839f85a22)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:23.363661 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32b926-9b56-4747-ba1d-6f874ba604c0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:23.367114 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 137/269] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:23 2022] GET /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:23.373107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32b926-9b56-4747-ba1d-6f874ba604c0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:23.468957 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:23.486904 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 133/270] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:23.532041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89e4ed-ac4d-40a7-8bca-dc6fcd86e98a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:23.532355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89e4ed-ac4d-40a7-8bca-dc6fcd86e98a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:23.532673 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89e4ed-ac4d-40a7-8bca-dc6fcd86e98a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:23.534160 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 134/271] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:23.535565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89e4ed-ac4d-40a7-8bca-dc6fcd86e98a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:23.555085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32b926-9b56-4747-ba1d-6f874ba604c0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:23.600233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01076a08-2421-4d67-ad9a-4883fa2eac72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:23.601743 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01076a08-2421-4d67-ad9a-4883fa2eac72 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:23.602032 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01076a08-2421-4d67-ad9a-4883fa2eac72 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:23.602571 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01076a08-2421-4d67-ad9a-4883fa2eac72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:23.604080 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01076a08-2421-4d67-ad9a-4883fa2eac72 None admin] RBAC: Adding query filter params (role.id=8c039234db9640d0bdc0c2a629ff9e52, scope.project.id=57b2ea019ba3413bafb7714839f85a22, user.id=b8bc8c0a6f2640f69272813cdd67ca2e) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:23.604397 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01076a08-2421-4d67-ad9a-4883fa2eac72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:23.605892 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01076a08-2421-4d67-ad9a-4883fa2eac72 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:23.615061 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 138/272] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:23 2022] GET /identity/v3/role_assignments?user.id=b8bc8c0a6f2640f69272813cdd67ca2e&scope.project.id=57b2ea019ba3413bafb7714839f85a22&role.id=8c039234db9640d0bdc0c2a629ff9e52 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:23.617763 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38db0e-3448-4f7d-ae15-32625da29836 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:23.618051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38db0e-3448-4f7d-ae15-32625da29836 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:23.618327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38db0e-3448-4f7d-ae15-32625da29836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:23.629707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:23.647259 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bd38db0e-3448-4f7d-ae15-32625da29836 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:23.648873 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bd38db0e-3448-4f7d-ae15-32625da29836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:23.711441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89e4ed-ac4d-40a7-8bca-dc6fcd86e98a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:23.766343 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 139/273] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:23.791099 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38db0e-3448-4f7d-ae15-32625da29836 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:23.800377 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f119a11-cff9-421b-821f-4594147dd70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:23.803377 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f119a11-cff9-421b-821f-4594147dd70e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:23.805636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f119a11-cff9-421b-821f-4594147dd70e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:23.805915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f119a11-cff9-421b-821f-4594147dd70e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:23.807794 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f119a11-cff9-421b-821f-4594147dd70e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:23.809275 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f119a11-cff9-421b-821f-4594147dd70e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:23.813067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 135/274] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:23.816300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f119a11-cff9-421b-821f-4594147dd70e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:23.904865 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:23.921110 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38db0e-3448-4f7d-ae15-32625da29836 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:23.956856 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f119a11-cff9-421b-821f-4594147dd70e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:23.964237 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ae4e7-543d-4ab0-b618-1d4a1370fdab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:23.965336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ae4e7-543d-4ab0-b618-1d4a1370fdab None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:23.965873 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ae4e7-543d-4ab0-b618-1d4a1370fdab None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:23.966364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ae4e7-543d-4ab0-b618-1d4a1370fdab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:23.968825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:23.985687 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ae4e7-543d-4ab0-b618-1d4a1370fdab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:23.994810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-229ae4e7-543d-4ab0-b618-1d4a1370fdab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:23.994810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:23.994810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:23.994810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:23.994810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:23.994810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:23.994810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:23.994810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:23.994810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:23.994810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:23.994810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:23.994810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:23.994810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:23.994810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:23.994810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:23.994810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:23.995700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:23.995700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:23.995700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:23.995700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:23.995700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:23.995700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:23.995700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:23.995700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:23.995700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:23.995700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:23.995700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:23.995700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:23.995700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:23.995700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:23.995700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:23.995700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:23.996636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:23.996636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:23.996636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:23.996636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:23.996636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:23.996636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:23.996636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:23.996636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:23.996636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:23.996636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:23.996636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:23.996636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:23.999009 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 140/275] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:24.003140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9cf70-51d8-4187-b811-25c12cc06897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:24.003140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9cf70-51d8-4187-b811-25c12cc06897 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:24.003140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9cf70-51d8-4187-b811-25c12cc06897 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:24.003140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9cf70-51d8-4187-b811-25c12cc06897 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:24.003462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9cf70-51d8-4187-b811-25c12cc06897 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:24.004848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ae4e7-543d-4ab0-b618-1d4a1370fdab None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:24.005136 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9cf70-51d8-4187-b811-25c12cc06897 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:24.031826 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 136/276] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 18 10:15:23 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:15:24.141605 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ae4e7-543d-4ab0-b618-1d4a1370fdab None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:24.185474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ed94a-7500-48f1-abe9-2908b9814132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:24.187031 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ed94a-7500-48f1-abe9-2908b9814132 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:24.187528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ed94a-7500-48f1-abe9-2908b9814132 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:24.188048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ed94a-7500-48f1-abe9-2908b9814132 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:24.189059 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ed94a-7500-48f1-abe9-2908b9814132 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:24.189561 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ed94a-7500-48f1-abe9-2908b9814132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:24.191378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ed94a-7500-48f1-abe9-2908b9814132 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:24.201324 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 141/277] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:24.207385 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9cf70-51d8-4187-b811-25c12cc06897 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:24.326366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9cf70-51d8-4187-b811-25c12cc06897 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:24.341557 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:24.363990 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69318d9e-acb2-4dd6-91f7-1ce61d6512a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:24.364925 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69318d9e-acb2-4dd6-91f7-1ce61d6512a1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:24.365211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69318d9e-acb2-4dd6-91f7-1ce61d6512a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:24.365496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69318d9e-acb2-4dd6-91f7-1ce61d6512a1 None admin] PATH_INFO: `/v3/users/47a5e83d849a4887a3c3ce81bac72053` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:24.380293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69318d9e-acb2-4dd6-91f7-1ce61d6512a1 None admin] RBAC: Authorizing `identity:get_user(user_id=47a5e83d849a4887a3c3ce81bac72053)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:24.381840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69318d9e-acb2-4dd6-91f7-1ce61d6512a1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:24.384203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 137/278] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:24 2022] GET /identity/v3/users/47a5e83d849a4887a3c3ce81bac72053 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:24.389965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ed94a-7500-48f1-abe9-2908b9814132 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:24.449709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:24.496376 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ed94a-7500-48f1-abe9-2908b9814132 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:24.531455 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e257f-bcef-47bc-8e59-11c3ab2faba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:24.532121 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e257f-bcef-47bc-8e59-11c3ab2faba2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:24.532411 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e257f-bcef-47bc-8e59-11c3ab2faba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:24.532723 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e257f-bcef-47bc-8e59-11c3ab2faba2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:24.533575 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e257f-bcef-47bc-8e59-11c3ab2faba2 None admin] RBAC: Adding query filter params (role.id=2820090979cb487db491f835e03e6471, user.id=47a5e83d849a4887a3c3ce81bac72053, scope.system=all) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:24.533894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e257f-bcef-47bc-8e59-11c3ab2faba2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:24.535357 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e257f-bcef-47bc-8e59-11c3ab2faba2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:24.541559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 142/279] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Jul 18 10:15:24 2022] GET /identity/v3/role_assignments?user.id=47a5e83d849a4887a3c3ce81bac72053&scope.system=all&role.id=2820090979cb487db491f835e03e6471 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:24.583519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c81eae-d404-4a74-a4fd-2935c3fcc3ec None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:24.583798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c81eae-d404-4a74-a4fd-2935c3fcc3ec None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:24.584061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c81eae-d404-4a74-a4fd-2935c3fcc3ec None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:24.586399 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 138/280] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:24.591956 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b4417-3ceb-4a0f-9ae7-b5dcd05b0dd0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:24.592255 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b4417-3ceb-4a0f-9ae7-b5dcd05b0dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:24.592573 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b4417-3ceb-4a0f-9ae7-b5dcd05b0dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:24.611791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bb3b4417-3ceb-4a0f-9ae7-b5dcd05b0dd0 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:24.613071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bb3b4417-3ceb-4a0f-9ae7-b5dcd05b0dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:24.714780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 143/281] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:24.753274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c81eae-d404-4a74-a4fd-2935c3fcc3ec None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:24.881986 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c81eae-d404-4a74-a4fd-2935c3fcc3ec None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:24.919043 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1529902-9848-47f8-8cdb-dd99ea545908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:24.919850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1529902-9848-47f8-8cdb-dd99ea545908 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:24.920133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1529902-9848-47f8-8cdb-dd99ea545908 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:24.920409 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1529902-9848-47f8-8cdb-dd99ea545908 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:24.931624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1529902-9848-47f8-8cdb-dd99ea545908 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:24.939516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e1529902-9848-47f8-8cdb-dd99ea545908 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:15:24.939516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:24.939516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:24.939516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:24.939516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:24.939516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:24.939516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:24.939516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:24.939516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:24.939516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:24.939516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:24.939516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:24.939516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:24.939516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:24.939516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:24.939516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:24.940414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:24.940414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:24.940414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:24.940414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:24.940414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:24.940414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:24.940414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:24.940414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:24.940414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:24.940414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:24.940414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:24.940414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:24.940414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:24.940414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:24.940414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:24.940414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:24.941537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:24.941537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:24.941537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:24.941537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:24.941537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:24.941537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:24.941537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:24.941537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:24.941537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:24.941537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:24.941537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:15:24.941537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:24.941537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 139/282] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:24.947176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b4417-3ceb-4a0f-9ae7-b5dcd05b0dd0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:25.015661 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:25.082354 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b4417-3ceb-4a0f-9ae7-b5dcd05b0dd0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:25.121639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04ce42-6d04-438c-a642-22f8bbd84b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:25.122316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04ce42-6d04-438c-a642-22f8bbd84b0a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:25.122600 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04ce42-6d04-438c-a642-22f8bbd84b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:25.122876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04ce42-6d04-438c-a642-22f8bbd84b0a None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:25.123628 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04ce42-6d04-438c-a642-22f8bbd84b0a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:25.123935 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04ce42-6d04-438c-a642-22f8bbd84b0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:25.125352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04ce42-6d04-438c-a642-22f8bbd84b0a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:25.134852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 144/283] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:25.140068 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1529902-9848-47f8-8cdb-dd99ea545908 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:25.195601 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:25.289315 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1529902-9848-47f8-8cdb-dd99ea545908 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:25.323429 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d9f8f-8450-44ce-ae00-7664fd38a766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:25.324303 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d9f8f-8450-44ce-ae00-7664fd38a766 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:25.324605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d9f8f-8450-44ce-ae00-7664fd38a766 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:25.324880 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d9f8f-8450-44ce-ae00-7664fd38a766 None admin] PATH_INFO: `/v3/users/8a57c25e3f0141079354e411060bfd18` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:25.326060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d9f8f-8450-44ce-ae00-7664fd38a766 None admin] RBAC: Authorizing `identity:get_user(user_id=8a57c25e3f0141079354e411060bfd18)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:25.327532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d9f8f-8450-44ce-ae00-7664fd38a766 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:25.329194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 140/284] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:25 2022] GET /identity/v3/users/8a57c25e3f0141079354e411060bfd18 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:25.334824 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04ce42-6d04-438c-a642-22f8bbd84b0a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:25.491403 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04ce42-6d04-438c-a642-22f8bbd84b0a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:25.508129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:25.537242 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff502956-ab3f-4748-94e1-7de8c1658a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:25.538138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff502956-ab3f-4748-94e1-7de8c1658a1f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:25.538428 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff502956-ab3f-4748-94e1-7de8c1658a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:25.538707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff502956-ab3f-4748-94e1-7de8c1658a1f None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:25.539913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff502956-ab3f-4748-94e1-7de8c1658a1f None admin] RBAC: Authorizing `identity:get_project(project_id=57b2ea019ba3413bafb7714839f85a22)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:25.541407 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff502956-ab3f-4748-94e1-7de8c1658a1f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:25.543120 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 145/285] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:25 2022] GET /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:25.549352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff502956-ab3f-4748-94e1-7de8c1658a1f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:25.550238 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aec76e-80e4-4216-82ef-012120fd26d0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:25.550535 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aec76e-80e4-4216-82ef-012120fd26d0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:25.550819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aec76e-80e4-4216-82ef-012120fd26d0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:25.552421 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 141/286] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:25.558043 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c477562-1e80-465e-a1ed-0acd2939ed9a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:25.558379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c477562-1e80-465e-a1ed-0acd2939ed9a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:25.558663 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c477562-1e80-465e-a1ed-0acd2939ed9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:25.576496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2c477562-1e80-465e-a1ed-0acd2939ed9a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:25.577938 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2c477562-1e80-465e-a1ed-0acd2939ed9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:25.658086 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 142/287] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:25.711858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff502956-ab3f-4748-94e1-7de8c1658a1f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:25.716840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c477562-1e80-465e-a1ed-0acd2939ed9a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:25.766008 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:25.781769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0f17c-b2ac-478e-8538-37bf5439f059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:25.782662 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0f17c-b2ac-478e-8538-37bf5439f059 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:25.782942 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0f17c-b2ac-478e-8538-37bf5439f059 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:25.783215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0f17c-b2ac-478e-8538-37bf5439f059 None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22/users/8a57c25e3f0141079354e411060bfd18/roles/e6c6a363f7034bd0a72b72ffe324aab8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:25.785408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0f17c-b2ac-478e-8538-37bf5439f059 None admin] RBAC: Authorizing `identity:create_grant(project_id=57b2ea019ba3413bafb7714839f85a22, user_id=8a57c25e3f0141079354e411060bfd18, role_id=e6c6a363f7034bd0a72b72ffe324aab8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:25.786868 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0f17c-b2ac-478e-8538-37bf5439f059 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:25.823737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 146/288] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:15:25 2022] PUT /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22/users/8a57c25e3f0141079354e411060bfd18/roles/e6c6a363f7034bd0a72b72ffe324aab8 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:15:25.886539 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c477562-1e80-465e-a1ed-0acd2939ed9a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:25.944195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd3ded-8691-42ef-a356-b69e8870c9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:25.945653 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd3ded-8691-42ef-a356-b69e8870c9cb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:25.945935 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd3ded-8691-42ef-a356-b69e8870c9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:25.946210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd3ded-8691-42ef-a356-b69e8870c9cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:25.962537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd3ded-8691-42ef-a356-b69e8870c9cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:25.970469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f9fd3ded-8691-42ef-a356-b69e8870c9cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:25.970469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:25.970469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:25.970469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:25.970469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:25.970469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:25.970469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:25.970469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:25.970469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:25.970469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:25.970469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:25.970469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:25.970469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:25.970469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:25.970469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:25.970469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:25.971413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:25.971413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:25.971413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:25.971413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:25.971413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:25.971413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:25.971413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:25.971413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:25.971413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:25.971413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:25.971413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:25.971413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:25.971413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:25.971413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:25.971413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:25.971413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:25.972212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:25.972212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:25.972212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:25.972212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:25.972212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:25.972212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:25.972212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:25.972212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:25.972212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:25.972212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:25.972212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:25.972212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:25.972212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 143/289] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:15:25.981684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0f17c-b2ac-478e-8538-37bf5439f059 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:26.046005 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:26.127287 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0f17c-b2ac-478e-8538-37bf5439f059 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:26.165269 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55024a7-529b-4e71-aabe-f2b10f3ae9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:26.165948 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55024a7-529b-4e71-aabe-f2b10f3ae9cd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:26.166234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55024a7-529b-4e71-aabe-f2b10f3ae9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:26.166511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55024a7-529b-4e71-aabe-f2b10f3ae9cd None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:26.167262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55024a7-529b-4e71-aabe-f2b10f3ae9cd None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:26.167571 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55024a7-529b-4e71-aabe-f2b10f3ae9cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:26.168977 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55024a7-529b-4e71-aabe-f2b10f3ae9cd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:26.184660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 147/290] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:26.193619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd3ded-8691-42ef-a356-b69e8870c9cb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:26.326230 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:26.337222 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd3ded-8691-42ef-a356-b69e8870c9cb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:26.388162 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6eb7d-3e0f-46e8-9a3a-8c707ada3abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:26.389075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6eb7d-3e0f-46e8-9a3a-8c707ada3abf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:26.389358 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6eb7d-3e0f-46e8-9a3a-8c707ada3abf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:26.389638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6eb7d-3e0f-46e8-9a3a-8c707ada3abf None admin] PATH_INFO: `/v3/users/d87b27ff26c54cbb843d087906fce4ba` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:26.390843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d6eb7d-3e0f-46e8-9a3a-8c707ada3abf None admin] RBAC: Authorizing `identity:get_user(user_id=d87b27ff26c54cbb843d087906fce4ba)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:26.392305 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d6eb7d-3e0f-46e8-9a3a-8c707ada3abf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:26.394290 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 144/291] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:26 2022] GET /identity/v3/users/d87b27ff26c54cbb843d087906fce4ba => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:26.400131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55024a7-529b-4e71-aabe-f2b10f3ae9cd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:26.526393 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55024a7-529b-4e71-aabe-f2b10f3ae9cd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:26.528047 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:26.577611 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95e358-d829-48a9-b770-a0d04002d463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:26.578231 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95e358-d829-48a9-b770-a0d04002d463 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:26.578518 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95e358-d829-48a9-b770-a0d04002d463 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:26.578795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95e358-d829-48a9-b770-a0d04002d463 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:26.663521 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95e358-d829-48a9-b770-a0d04002d463 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:26.665055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95e358-d829-48a9-b770-a0d04002d463 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:26.797354 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 148/292] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:26.805619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6eb7d-3e0f-46e8-9a3a-8c707ada3abf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:26.889048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48ae2a-8fc8-4ff3-bd84-489670e4df75 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:26.889397 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48ae2a-8fc8-4ff3-bd84-489670e4df75 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:26.889614 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48ae2a-8fc8-4ff3-bd84-489670e4df75 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:26.891006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 149/293] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:26.902554 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103c7a0-5311-4f6d-89b6-5123c51c32f1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:26.902845 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103c7a0-5311-4f6d-89b6-5123c51c32f1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:26.903129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103c7a0-5311-4f6d-89b6-5123c51c32f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:26.916696 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:26.940837 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6eb7d-3e0f-46e8-9a3a-8c707ada3abf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:26.946198 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1103c7a0-5311-4f6d-89b6-5123c51c32f1 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:26.947415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1103c7a0-5311-4f6d-89b6-5123c51c32f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:26.976471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864e9e4-f5d7-444a-8065-8ed5e9fb997e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:26.977718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864e9e4-f5d7-444a-8065-8ed5e9fb997e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:26.978194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864e9e4-f5d7-444a-8065-8ed5e9fb997e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:26.978921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864e9e4-f5d7-444a-8065-8ed5e9fb997e None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:26.980297 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864e9e4-f5d7-444a-8065-8ed5e9fb997e None admin] RBAC: Authorizing `identity:get_project(project_id=57b2ea019ba3413bafb7714839f85a22)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:26.982359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864e9e4-f5d7-444a-8065-8ed5e9fb997e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:26.984706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 145/294] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:26 2022] GET /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:26.986154 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ed2db-154e-4ba5-b844-dbdf6c4d734b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:26.986637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ed2db-154e-4ba5-b844-dbdf6c4d734b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:26.987104 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ed2db-154e-4ba5-b844-dbdf6c4d734b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:26.988755 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 146/295] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:26.990531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ed2db-154e-4ba5-b844-dbdf6c4d734b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:27.023075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:27.029297 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 150/296] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:27.030772 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59586f48-5ea2-4201-bb54-44f66060ad8c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:27.031189 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59586f48-5ea2-4201-bb54-44f66060ad8c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:27.031619 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59586f48-5ea2-4201-bb54-44f66060ad8c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:27.033041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 151/297] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:27.034187 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d055d8-f032-4e07-bf81-1254b73f0b44 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:27.034600 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d055d8-f032-4e07-bf81-1254b73f0b44 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:27.035002 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d055d8-f032-4e07-bf81-1254b73f0b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:27.052889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-71d055d8-f032-4e07-bf81-1254b73f0b44 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:27.054385 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-71d055d8-f032-4e07-bf81-1254b73f0b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:27.104914 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ed2db-154e-4ba5-b844-dbdf6c4d734b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:27.126828 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 152/298] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:27.128314 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760f82b-ce54-4cca-826a-ebdaf87302da None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:27.128642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760f82b-ce54-4cca-826a-ebdaf87302da None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:27.128922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760f82b-ce54-4cca-826a-ebdaf87302da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:27.150476 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5760f82b-ce54-4cca-826a-ebdaf87302da None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:27.151858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5760f82b-ce54-4cca-826a-ebdaf87302da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:27.156478 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1bbc6-10d0-41bf-9523-846d6f39aa92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:27.157191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1bbc6-10d0-41bf-9523-846d6f39aa92 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:27.157482 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1bbc6-10d0-41bf-9523-846d6f39aa92 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:27.157766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1bbc6-10d0-41bf-9523-846d6f39aa92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:27.158621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1bbc6-10d0-41bf-9523-846d6f39aa92 None admin] RBAC: Adding query filter params (role.id=2820090979cb487db491f835e03e6471, scope.project.id=57b2ea019ba3413bafb7714839f85a22, user.id=d87b27ff26c54cbb843d087906fce4ba) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:27.158942 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1bbc6-10d0-41bf-9523-846d6f39aa92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:27.160405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1bbc6-10d0-41bf-9523-846d6f39aa92 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:27.166834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 147/299] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:26 2022] GET /identity/v3/role_assignments?user.id=d87b27ff26c54cbb843d087906fce4ba&scope.project.id=57b2ea019ba3413bafb7714839f85a22&role.id=2820090979cb487db491f835e03e6471 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:27.168064 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1bbc6-10d0-41bf-9523-846d6f39aa92 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:27.244177 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:27.262505 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 153/300] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:27.263726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760f82b-ce54-4cca-826a-ebdaf87302da None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:27.310922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:27.324260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1bbc6-10d0-41bf-9523-846d6f39aa92 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:27.371768 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5f9ab-a92b-4565-abda-643f50f7716b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:27.372571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5f9ab-a92b-4565-abda-643f50f7716b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:27.372778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5f9ab-a92b-4565-abda-643f50f7716b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:27.373058 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5f9ab-a92b-4565-abda-643f50f7716b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:27.390346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5f9ab-a92b-4565-abda-643f50f7716b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:27.399047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-88d5f9ab-a92b-4565-abda-643f50f7716b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:15:27.399047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:27.399047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:27.399047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:27.399047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:27.399047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:27.399047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:27.399047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:27.399047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:27.399047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:27.399047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:27.399047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:27.399047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:27.399047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:27.399047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:27.399047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:27.399952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:27.399952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:27.399952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:27.399952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:27.399952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:27.399952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:27.399952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:27.399952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:27.399952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:27.399952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:27.399952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:27.399952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:27.399952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:27.399952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:27.399952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:27.399952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:27.400800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:27.400800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:27.400800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:27.400800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:27.400800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:27.400800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:27.400800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:27.400800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:27.400800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:27.400800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:27.400800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:15:27.400800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:27.408817 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 148/301] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:27.408817 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5f9ab-a92b-4565-abda-643f50f7716b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:27.410503 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760f82b-ce54-4cca-826a-ebdaf87302da None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:27.450714 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d39245-3678-4d31-a95c-cbaeb45f152c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:27.451872 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d39245-3678-4d31-a95c-cbaeb45f152c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:27.452169 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d39245-3678-4d31-a95c-cbaeb45f152c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:27.452452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d39245-3678-4d31-a95c-cbaeb45f152c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:27.467809 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d39245-3678-4d31-a95c-cbaeb45f152c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:27.476008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-59d39245-3678-4d31-a95c-cbaeb45f152c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:27.476008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:27.476008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:27.476008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:27.476008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:27.476008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:27.476008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:27.476008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:27.476008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:27.476008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:27.476008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:27.476008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:27.476008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:27.476008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:27.476008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:27.476008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:27.477034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:27.477034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:27.477034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:27.477034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:27.477034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:27.477034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:27.477034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:27.477034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:27.477034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:27.477034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:27.477034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:27.477034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:27.477034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:27.477034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:27.477034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:27.477034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:27.477850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:27.477850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:27.477850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:27.477850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:27.477850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:27.477850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:27.477850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:27.477850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:27.477850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:27.477850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:27.477850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:27.477850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:27.477850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 154/302] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:27.481365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d39245-3678-4d31-a95c-cbaeb45f152c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:27.519023 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5f9ab-a92b-4565-abda-643f50f7716b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:27.537872 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:27.553454 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ebcbb-c6d4-4b19-a0ef-68b7c6ebc38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:27.554912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ebcbb-c6d4-4b19-a0ef-68b7c6ebc38d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:27.555260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ebcbb-c6d4-4b19-a0ef-68b7c6ebc38d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:27.555606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ebcbb-c6d4-4b19-a0ef-68b7c6ebc38d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:27.556795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ebcbb-c6d4-4b19-a0ef-68b7c6ebc38d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:27.558252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ebcbb-c6d4-4b19-a0ef-68b7c6ebc38d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:27.559859 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 149/303] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:27.560712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ebcbb-c6d4-4b19-a0ef-68b7c6ebc38d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:27.583546 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:27.592757 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d39245-3678-4d31-a95c-cbaeb45f152c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:27.629978 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888ee6d-976d-497c-b473-241cf4569faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:27.630894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888ee6d-976d-497c-b473-241cf4569faa None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:27.631377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888ee6d-976d-497c-b473-241cf4569faa None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:27.631913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888ee6d-976d-497c-b473-241cf4569faa None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:27.632986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888ee6d-976d-497c-b473-241cf4569faa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:27.633361 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888ee6d-976d-497c-b473-241cf4569faa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:27.634750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888ee6d-976d-497c-b473-241cf4569faa None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:27.647883 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 155/304] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:27.647883 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888ee6d-976d-497c-b473-241cf4569faa None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:27.669987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ebcbb-c6d4-4b19-a0ef-68b7c6ebc38d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:27.707442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe89da6-56a3-4977-8d8e-ff2785587ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:27.708432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe89da6-56a3-4977-8d8e-ff2785587ffa None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:27.708916 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe89da6-56a3-4977-8d8e-ff2785587ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:27.709345 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe89da6-56a3-4977-8d8e-ff2785587ffa None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:27.710399 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe89da6-56a3-4977-8d8e-ff2785587ffa None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:27.710872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe89da6-56a3-4977-8d8e-ff2785587ffa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:27.712394 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe89da6-56a3-4977-8d8e-ff2785587ffa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:27.721718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 150/305] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:27.722684 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe89da6-56a3-4977-8d8e-ff2785587ffa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:27.753713 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888ee6d-976d-497c-b473-241cf4569faa None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:27.769316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:27.799833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054f81a-85c7-4b56-8945-fb44598f0811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:27.800533 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054f81a-85c7-4b56-8945-fb44598f0811 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:27.800831 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054f81a-85c7-4b56-8945-fb44598f0811 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:27.801113 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054f81a-85c7-4b56-8945-fb44598f0811 None admin] PATH_INFO: `/v3/groups` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:27.801909 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054f81a-85c7-4b56-8945-fb44598f0811 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:27.803373 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054f81a-85c7-4b56-8945-fb44598f0811 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:27.819947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:27.820751 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 156/306] 10.222.0.33 () {60 vars in 1134 bytes} [Mon Jul 18 10:15:27 2022] POST /identity/v3/groups => generated 229 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:15:27.822096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054f81a-85c7-4b56-8945-fb44598f0811 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:27.853148 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe89da6-56a3-4977-8d8e-ff2785587ffa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:27.894571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5a0f2-fa22-49d9-9d9e-1733e53152f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:27.895577 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5a0f2-fa22-49d9-9d9e-1733e53152f5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:27.895862 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5a0f2-fa22-49d9-9d9e-1733e53152f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:27.896140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5a0f2-fa22-49d9-9d9e-1733e53152f5 None admin] PATH_INFO: `/v3/users/2b246182e6a649fc9d2439e9d1badc66` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:27.912432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a5a0f2-fa22-49d9-9d9e-1733e53152f5 None admin] RBAC: Authorizing `identity:get_user(user_id=2b246182e6a649fc9d2439e9d1badc66)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:27.914122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a5a0f2-fa22-49d9-9d9e-1733e53152f5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:27.915925 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 151/307] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:27 2022] GET /identity/v3/users/2b246182e6a649fc9d2439e9d1badc66 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:27.925757 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5a0f2-fa22-49d9-9d9e-1733e53152f5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:27.960846 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054f81a-85c7-4b56-8945-fb44598f0811 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:28.011841 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52432daf-cb85-427c-8a09-3d1a2278e18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:28.013024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52432daf-cb85-427c-8a09-3d1a2278e18b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:28.013458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52432daf-cb85-427c-8a09-3d1a2278e18b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:28.013915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52432daf-cb85-427c-8a09-3d1a2278e18b None admin] PATH_INFO: `/v3/users/47a5e83d849a4887a3c3ce81bac72053` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:28.015182 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52432daf-cb85-427c-8a09-3d1a2278e18b None admin] RBAC: Authorizing `identity:get_user(user_id=47a5e83d849a4887a3c3ce81bac72053)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:28.017156 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52432daf-cb85-427c-8a09-3d1a2278e18b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:28.019380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 157/308] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:27 2022] GET /identity/v3/users/47a5e83d849a4887a3c3ce81bac72053 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:28.025454 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52432daf-cb85-427c-8a09-3d1a2278e18b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:28.043329 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5a0f2-fa22-49d9-9d9e-1733e53152f5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:28.075016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:28.091869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c280d-37f3-46e4-bb8e-3906ca269607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:28.092548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c280d-37f3-46e4-bb8e-3906ca269607 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:28.092832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c280d-37f3-46e4-bb8e-3906ca269607 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:28.093102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c280d-37f3-46e4-bb8e-3906ca269607 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:28.094230 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c280d-37f3-46e4-bb8e-3906ca269607 None admin] RBAC: Adding query filter params (role.id=e6c6a363f7034bd0a72b72ffe324aab8, user.id=2b246182e6a649fc9d2439e9d1badc66, scope.system=all) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:28.094230 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c280d-37f3-46e4-bb8e-3906ca269607 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:28.095698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c280d-37f3-46e4-bb8e-3906ca269607 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:28.101968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 152/309] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Jul 18 10:15:27 2022] GET /identity/v3/role_assignments?user.id=2b246182e6a649fc9d2439e9d1badc66&scope.system=all&role.id=e6c6a363f7034bd0a72b72ffe324aab8 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:28.160545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52432daf-cb85-427c-8a09-3d1a2278e18b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:28.201267 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff3a03-d3ca-4431-9708-4dd9396c7002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:28.202177 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff3a03-d3ca-4431-9708-4dd9396c7002 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:28.202459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff3a03-d3ca-4431-9708-4dd9396c7002 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:28.202806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff3a03-d3ca-4431-9708-4dd9396c7002 None admin] PATH_INFO: `/v3/system/users/47a5e83d849a4887a3c3ce81bac72053/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:28.204417 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff3a03-d3ca-4431-9708-4dd9396c7002 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=47a5e83d849a4887a3c3ce81bac72053, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:28.205887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff3a03-d3ca-4431-9708-4dd9396c7002 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:28.215966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 158/310] 10.222.0.33 () {58 vars in 1334 bytes} [Mon Jul 18 10:15:28 2022] PUT /identity/v3/system/users/47a5e83d849a4887a3c3ce81bac72053/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:15:28.263887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90fa1b-4327-43a9-9418-eba594160f40 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:28.264200 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90fa1b-4327-43a9-9418-eba594160f40 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:28.264462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90fa1b-4327-43a9-9418-eba594160f40 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:28.266013 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 153/311] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:28.272553 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58c81a-81be-4c83-be2b-e96f75413f33 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:28.273356 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58c81a-81be-4c83-be2b-e96f75413f33 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:28.273704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58c81a-81be-4c83-be2b-e96f75413f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:28.292822 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bd58c81a-81be-4c83-be2b-e96f75413f33 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:28.294111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bd58c81a-81be-4c83-be2b-e96f75413f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:28.373166 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 159/312] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:28.396533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90fa1b-4327-43a9-9418-eba594160f40 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:28.401714 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:28.509902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90fa1b-4327-43a9-9418-eba594160f40 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:28.545757 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546503cb-a3c9-4753-8d47-23af31d69505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:28.546555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546503cb-a3c9-4753-8d47-23af31d69505 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:28.546838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546503cb-a3c9-4753-8d47-23af31d69505 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:28.547112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546503cb-a3c9-4753-8d47-23af31d69505 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:28.558352 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546503cb-a3c9-4753-8d47-23af31d69505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:28.566048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-546503cb-a3c9-4753-8d47-23af31d69505 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:15:28.566048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:28.566048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:28.566048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:28.566048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:28.566048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:28.566048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:28.566048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:28.566048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:28.566048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:28.566048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:28.566048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:28.566048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:28.566048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:28.566048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:28.566048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:28.566920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:28.566920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:28.566920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:28.566920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:28.566920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:28.566920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:28.566920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:28.566920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:28.566920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:28.566920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:28.566920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:28.566920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:28.566920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:28.566920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:28.566920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:28.566920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:28.567695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:28.567695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:28.567695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:28.567695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:28.567695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:28.567695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:28.567695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:28.567695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:28.567695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:28.567695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:28.567695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:15:28.567695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:28.567695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 154/313] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:28.574386 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58c81a-81be-4c83-be2b-e96f75413f33 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:28.611589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:28.681811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58c81a-81be-4c83-be2b-e96f75413f33 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:28.716921 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e3493-59c4-4315-8bc3-586dffa9620e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:28.717923 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e3493-59c4-4315-8bc3-586dffa9620e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:28.718403 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e3493-59c4-4315-8bc3-586dffa9620e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:28.718896 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e3493-59c4-4315-8bc3-586dffa9620e None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:28.719836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e3493-59c4-4315-8bc3-586dffa9620e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:28.720335 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e3493-59c4-4315-8bc3-586dffa9620e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:28.721913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e3493-59c4-4315-8bc3-586dffa9620e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:28.731461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 160/314] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:28.737874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546503cb-a3c9-4753-8d47-23af31d69505 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:28.801729 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:28.847847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546503cb-a3c9-4753-8d47-23af31d69505 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:28.885850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b9ecf-baf9-4db8-baab-71b2f42836da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:28.886728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b9ecf-baf9-4db8-baab-71b2f42836da None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:28.887010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b9ecf-baf9-4db8-baab-71b2f42836da None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:28.887288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b9ecf-baf9-4db8-baab-71b2f42836da None admin] PATH_INFO: `/v3/users/8a57c25e3f0141079354e411060bfd18` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:28.888491 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b9ecf-baf9-4db8-baab-71b2f42836da None admin] RBAC: Authorizing `identity:get_user(user_id=8a57c25e3f0141079354e411060bfd18)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:28.889963 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b9ecf-baf9-4db8-baab-71b2f42836da None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:28.892011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 155/315] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:28 2022] GET /identity/v3/users/8a57c25e3f0141079354e411060bfd18 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:28.897298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e3493-59c4-4315-8bc3-586dffa9620e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:28.999712 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:29.003748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e3493-59c4-4315-8bc3-586dffa9620e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:29.039728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c864415-e238-486d-9faf-829374e69d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:29.040382 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c864415-e238-486d-9faf-829374e69d1c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:29.040697 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c864415-e238-486d-9faf-829374e69d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:29.040976 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c864415-e238-486d-9faf-829374e69d1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:29.117651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c864415-e238-486d-9faf-829374e69d1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:29.119131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c864415-e238-486d-9faf-829374e69d1c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:29.229123 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:29.236593 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 161/316] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:29.238173 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b9ecf-baf9-4db8-baab-71b2f42836da None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:29.346424 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b9ecf-baf9-4db8-baab-71b2f42836da None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:29.375424 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:29.382520 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf5833-06f9-4058-b146-8d390bc80a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:29.383395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf5833-06f9-4058-b146-8d390bc80a9c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:29.383681 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf5833-06f9-4058-b146-8d390bc80a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:29.384244 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf5833-06f9-4058-b146-8d390bc80a9c None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:29.385493 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf5833-06f9-4058-b146-8d390bc80a9c None admin] RBAC: Authorizing `identity:get_project(project_id=57b2ea019ba3413bafb7714839f85a22)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:29.386958 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf5833-06f9-4058-b146-8d390bc80a9c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:29.388718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 156/317] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:29 2022] GET /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:29.397934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c864415-e238-486d-9faf-829374e69d1c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:29.501168 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c864415-e238-486d-9faf-829374e69d1c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:29.536313 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75810465-6378-4df8-a827-8f9030580e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:29.536998 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75810465-6378-4df8-a827-8f9030580e9b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:29.537288 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75810465-6378-4df8-a827-8f9030580e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:29.537570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75810465-6378-4df8-a827-8f9030580e9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:29.538620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75810465-6378-4df8-a827-8f9030580e9b None admin] RBAC: Adding query filter params (role.id=e6c6a363f7034bd0a72b72ffe324aab8, scope.project.id=57b2ea019ba3413bafb7714839f85a22, user.id=8a57c25e3f0141079354e411060bfd18) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:29.538942 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75810465-6378-4df8-a827-8f9030580e9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:29.540469 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75810465-6378-4df8-a827-8f9030580e9b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:29.546481 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 162/318] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:29 2022] GET /identity/v3/role_assignments?user.id=8a57c25e3f0141079354e411060bfd18&scope.project.id=57b2ea019ba3413bafb7714839f85a22&role.id=e6c6a363f7034bd0a72b72ffe324aab8 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:30.094446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ac5ec-c9d5-4f51-88f4-f951dd140a9b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:30.094959 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ac5ec-c9d5-4f51-88f4-f951dd140a9b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:30.095035 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ac5ec-c9d5-4f51-88f4-f951dd140a9b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:30.096384 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 157/319] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:30.101561 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eadd26-5dbb-4c96-ba79-91e3c881e2d9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:30.101969 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eadd26-5dbb-4c96-ba79-91e3c881e2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:30.102143 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eadd26-5dbb-4c96-ba79-91e3c881e2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:30.120003 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-61eadd26-5dbb-4c96-ba79-91e3c881e2d9 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:30.121236 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-61eadd26-5dbb-4c96-ba79-91e3c881e2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:30.164380 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75398f16-aaa4-47d9-ab78-0bbbfac49725 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:30.164778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75398f16-aaa4-47d9-ab78-0bbbfac49725 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:30.165089 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75398f16-aaa4-47d9-ab78-0bbbfac49725 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:30.166619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 158/320] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:30.171322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66629c1d-368d-4501-aeb1-11a5a369cf7d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:30.171613 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66629c1d-368d-4501-aeb1-11a5a369cf7d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:30.171893 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66629c1d-368d-4501-aeb1-11a5a369cf7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:30.185211 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:30.191725 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-66629c1d-368d-4501-aeb1-11a5a369cf7d None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:30.193322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-66629c1d-368d-4501-aeb1-11a5a369cf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:30.196839 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 163/321] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:30.219286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eadd26-5dbb-4c96-ba79-91e3c881e2d9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:30.265177 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 159/322] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:30.288189 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66629c1d-368d-4501-aeb1-11a5a369cf7d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:30.330120 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eadd26-5dbb-4c96-ba79-91e3c881e2d9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:30.363467 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf5dd4-2a36-4a6b-95ff-89606b2be529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:30.364177 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf5dd4-2a36-4a6b-95ff-89606b2be529 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:30.364460 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf5dd4-2a36-4a6b-95ff-89606b2be529 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:30.364762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf5dd4-2a36-4a6b-95ff-89606b2be529 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:30.365924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf5dd4-2a36-4a6b-95ff-89606b2be529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:30.367874 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf5dd4-2a36-4a6b-95ff-89606b2be529 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:30.369530 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 164/323] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:30.370684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944912a6-3b6a-428c-b89f-e1841ec549fc None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:30.370971 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944912a6-3b6a-428c-b89f-e1841ec549fc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:30.371246 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944912a6-3b6a-428c-b89f-e1841ec549fc None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:30.372480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 165/324] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:30.374947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944912a6-3b6a-428c-b89f-e1841ec549fc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:30.400756 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66629c1d-368d-4501-aeb1-11a5a369cf7d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:30.426735 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:30.435054 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:30.435857 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae5a13-754e-498e-a393-af09406a0df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:30.436827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae5a13-754e-498e-a393-af09406a0df8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:30.437116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae5a13-754e-498e-a393-af09406a0df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:30.437395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae5a13-754e-498e-a393-af09406a0df8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:30.447552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae5a13-754e-498e-a393-af09406a0df8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:30.455114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c9ae5a13-754e-498e-a393-af09406a0df8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:15:30.455114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:30.455114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:30.455114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:30.455114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:30.455114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:30.455114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:30.455114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:30.455114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:30.455114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:30.455114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:30.455114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:30.455114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:30.455114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:30.455114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:30.455114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:30.456155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:30.456155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:30.456155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:30.456155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:30.456155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:30.456155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:30.456155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:30.456155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:30.456155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:30.456155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:30.456155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:30.456155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:30.456155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:30.456155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:30.456155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:30.456155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:30.462045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:30.462045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:30.462045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:30.462045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:30.462045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:30.462045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:30.462045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:30.462045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:30.462045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:30.462045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:30.462045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:15:30.462045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:30.462045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 160/325] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:30.462045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de512ba4-817f-4d35-be7e-ffcc3255f5f8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:30.462759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de512ba4-817f-4d35-be7e-ffcc3255f5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:30.462759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de512ba4-817f-4d35-be7e-ffcc3255f5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:30.474001 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-de512ba4-817f-4d35-be7e-ffcc3255f5f8 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:30.475193 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-de512ba4-817f-4d35-be7e-ffcc3255f5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:30.475406 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944912a6-3b6a-428c-b89f-e1841ec549fc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:30.510011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cef88-a39c-47e2-b966-8d1666908084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:30.510670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cef88-a39c-47e2-b966-8d1666908084 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:30.510951 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cef88-a39c-47e2-b966-8d1666908084 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:30.511228 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cef88-a39c-47e2-b966-8d1666908084 None admin] PATH_INFO: `/v3/groups` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:30.511916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041cef88-a39c-47e2-b966-8d1666908084 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:30.513381 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041cef88-a39c-47e2-b966-8d1666908084 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:30.526292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 166/326] 10.222.0.33 () {60 vars in 1134 bytes} [Mon Jul 18 10:15:30 2022] POST /identity/v3/groups => generated 226 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:15:30.527092 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cef88-a39c-47e2-b966-8d1666908084 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:30.544780 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 161/327] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:30.568579 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de512ba4-817f-4d35-be7e-ffcc3255f5f8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:30.646929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cef88-a39c-47e2-b966-8d1666908084 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:30.669117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:30.682143 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de512ba4-817f-4d35-be7e-ffcc3255f5f8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:30.688081 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08f697-022d-4f10-9a02-590ee9c50920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:30.688816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08f697-022d-4f10-9a02-590ee9c50920 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:30.689444 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08f697-022d-4f10-9a02-590ee9c50920 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:30.689444 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08f697-022d-4f10-9a02-590ee9c50920 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:30.690122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08f697-022d-4f10-9a02-590ee9c50920 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:30.691006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08f697-022d-4f10-9a02-590ee9c50920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:30.691799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08f697-022d-4f10-9a02-590ee9c50920 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:30.701029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 167/328] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:30.702970 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:30.706898 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08f697-022d-4f10-9a02-590ee9c50920 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:30.722405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec157671-5176-4b39-a8c7-ef0cc9f87f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:30.723211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec157671-5176-4b39-a8c7-ef0cc9f87f55 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:30.723500 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec157671-5176-4b39-a8c7-ef0cc9f87f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:30.723782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec157671-5176-4b39-a8c7-ef0cc9f87f55 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:30.735551 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec157671-5176-4b39-a8c7-ef0cc9f87f55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:30.744398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ec157671-5176-4b39-a8c7-ef0cc9f87f55 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:30.744398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:30.744398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:30.744398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:30.744398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:30.744398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:30.744398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:30.744398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:30.744398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:30.744398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:30.744398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:30.744398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:30.744398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:30.744398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:30.744398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:30.744398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:30.745475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:30.745475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:30.745475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:30.745475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:30.745475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:30.745475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:30.745475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:30.745475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:30.745475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:30.745475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:30.745475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:30.745475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:30.745475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:30.745475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:30.745475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:30.745475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:30.746496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:30.746496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:30.746496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:30.746496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:30.746496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:30.746496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:30.746496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:30.746496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:30.746496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:30.746496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:30.746496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:30.746496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:30.748888 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 162/329] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:30.754345 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec157671-5176-4b39-a8c7-ef0cc9f87f55 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:30.824864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08f697-022d-4f10-9a02-590ee9c50920 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:30.871494 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1275dc-13ac-4253-aee2-b1bff2528ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:30.872382 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1275dc-13ac-4253-aee2-b1bff2528ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:30.872690 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1275dc-13ac-4253-aee2-b1bff2528ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:30.872973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1275dc-13ac-4253-aee2-b1bff2528ea5 None admin] PATH_INFO: `/v3/users/2b246182e6a649fc9d2439e9d1badc66` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:30.874186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1275dc-13ac-4253-aee2-b1bff2528ea5 None admin] RBAC: Authorizing `identity:get_user(user_id=2b246182e6a649fc9d2439e9d1badc66)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:30.875642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1275dc-13ac-4253-aee2-b1bff2528ea5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:30.877478 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 168/330] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:30 2022] GET /identity/v3/users/2b246182e6a649fc9d2439e9d1badc66 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:30.885278 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1275dc-13ac-4253-aee2-b1bff2528ea5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:30.885656 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec157671-5176-4b39-a8c7-ef0cc9f87f55 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:30.925348 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949efc4-80ea-4afa-8c8d-036507c4d9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:30.925875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949efc4-80ea-4afa-8c8d-036507c4d9db None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:30.926162 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949efc4-80ea-4afa-8c8d-036507c4d9db None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:30.926439 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949efc4-80ea-4afa-8c8d-036507c4d9db None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:30.927198 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949efc4-80ea-4afa-8c8d-036507c4d9db None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:30.927562 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949efc4-80ea-4afa-8c8d-036507c4d9db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:30.930048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949efc4-80ea-4afa-8c8d-036507c4d9db None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:30.939516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 163/331] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:30.946737 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949efc4-80ea-4afa-8c8d-036507c4d9db None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:30.948923 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:31.008985 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:31.021356 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1275dc-13ac-4253-aee2-b1bff2528ea5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:31.062240 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690e8d0-3a40-493a-80d6-d5927e728f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:31.063109 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690e8d0-3a40-493a-80d6-d5927e728f6f None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:31.063391 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690e8d0-3a40-493a-80d6-d5927e728f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:31.063667 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690e8d0-3a40-493a-80d6-d5927e728f6f None admin] PATH_INFO: `/v3/system/users/2b246182e6a649fc9d2439e9d1badc66/roles/e6c6a363f7034bd0a72b72ffe324aab8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:31.065294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690e8d0-3a40-493a-80d6-d5927e728f6f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2b246182e6a649fc9d2439e9d1badc66, role_id=e6c6a363f7034bd0a72b72ffe324aab8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:31.066685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690e8d0-3a40-493a-80d6-d5927e728f6f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:31.075591 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 169/332] 10.222.0.33 () {58 vars in 1334 bytes} [Mon Jul 18 10:15:30 2022] PUT /identity/v3/system/users/2b246182e6a649fc9d2439e9d1badc66/roles/e6c6a363f7034bd0a72b72ffe324aab8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:15:31.084339 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949efc4-80ea-4afa-8c8d-036507c4d9db None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:31.134269 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f22ec-4a55-47dd-9e90-d1ea4d187ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:31.135347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f22ec-4a55-47dd-9e90-d1ea4d187ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:31.135629 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f22ec-4a55-47dd-9e90-d1ea4d187ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:31.135949 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f22ec-4a55-47dd-9e90-d1ea4d187ca2 None admin] PATH_INFO: `/v3/users/47a5e83d849a4887a3c3ce81bac72053` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:31.137357 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f22ec-4a55-47dd-9e90-d1ea4d187ca2 None admin] RBAC: Authorizing `identity:get_user(user_id=47a5e83d849a4887a3c3ce81bac72053)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:31.138897 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f22ec-4a55-47dd-9e90-d1ea4d187ca2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:31.140976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 164/333] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:30 2022] GET /identity/v3/users/47a5e83d849a4887a3c3ce81bac72053 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:31.147163 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690e8d0-3a40-493a-80d6-d5927e728f6f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:31.276705 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690e8d0-3a40-493a-80d6-d5927e728f6f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:31.292989 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:31.316748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc47c5-bcd7-45f8-95c7-78a58a0b0238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:31.317683 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc47c5-bcd7-45f8-95c7-78a58a0b0238 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:31.318112 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc47c5-bcd7-45f8-95c7-78a58a0b0238 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:31.318601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc47c5-bcd7-45f8-95c7-78a58a0b0238 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:31.320116 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc47c5-bcd7-45f8-95c7-78a58a0b0238 None admin] RBAC: Adding query filter params (role.id=2820090979cb487db491f835e03e6471, user.id=47a5e83d849a4887a3c3ce81bac72053, scope.system=all) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:31.320603 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc47c5-bcd7-45f8-95c7-78a58a0b0238 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:31.322222 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc47c5-bcd7-45f8-95c7-78a58a0b0238 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:31.337616 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 170/334] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Jul 18 10:15:31 2022] GET /identity/v3/role_assignments?user.id=47a5e83d849a4887a3c3ce81bac72053&scope.system=all&role.id=2820090979cb487db491f835e03e6471 => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:33.095875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23b39a-e773-4cb9-9ffd-6299de09924b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:33.096400 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23b39a-e773-4cb9-9ffd-6299de09924b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:33.096400 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23b39a-e773-4cb9-9ffd-6299de09924b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:33.097957 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 165/335] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:33.114054 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f7f97-8c23-4d72-9a2b-c31d741c79d2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:33.114969 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f7f97-8c23-4d72-9a2b-c31d741c79d2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:33.115451 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f7f97-8c23-4d72-9a2b-c31d741c79d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:33.141519 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6b9f7f97-8c23-4d72-9a2b-c31d741c79d2 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:33.143090 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6b9f7f97-8c23-4d72-9a2b-c31d741c79d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:33.235532 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 171/336] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:33.278394 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23b39a-e773-4cb9-9ffd-6299de09924b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:33.408695 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774fd16-e568-4710-9de0-ed63af786f95 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:33.409398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774fd16-e568-4710-9de0-ed63af786f95 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:33.409882 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774fd16-e568-4710-9de0-ed63af786f95 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:33.410119 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23b39a-e773-4cb9-9ffd-6299de09924b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:33.411917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 172/337] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:33.416829 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff33acd-0701-4db5-b35e-b80b9cfe2fde None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:33.417118 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff33acd-0701-4db5-b35e-b80b9cfe2fde None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:33.417395 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff33acd-0701-4db5-b35e-b80b9cfe2fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:33.420280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:33.454475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14784e-7d49-44ab-8bd5-ea2f1317e08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:33.455290 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14784e-7d49-44ab-8bd5-ea2f1317e08d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:33.455650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14784e-7d49-44ab-8bd5-ea2f1317e08d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:33.455946 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14784e-7d49-44ab-8bd5-ea2f1317e08d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:33.461580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6ff33acd-0701-4db5-b35e-b80b9cfe2fde None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:33.462793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6ff33acd-0701-4db5-b35e-b80b9cfe2fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:33.467805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14784e-7d49-44ab-8bd5-ea2f1317e08d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:33.477459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-6f14784e-7d49-44ab-8bd5-ea2f1317e08d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:33.477459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:33.477459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:33.477459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:33.477459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:33.477459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:33.477459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:33.477459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:33.477459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:33.477459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:33.477459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:33.477459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:33.477459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:33.477459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:33.477459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:33.477459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:33.478588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:33.478588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:33.478588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:33.478588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:33.478588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:33.478588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:33.478588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:33.478588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:33.478588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:33.478588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:33.478588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:33.478588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:33.478588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:33.478588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:33.478588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:33.478588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:33.479863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:33.479863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:33.479863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:33.479863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:33.479863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:33.479863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:33.479863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:33.479863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:33.479863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:33.479863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:33.479863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:33.479863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:33.482215 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 166/338] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:15:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:33.484533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad82f72-1d71-4e46-868b-b77730391a76 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:33.484974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad82f72-1d71-4e46-868b-b77730391a76 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:33.485391 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad82f72-1d71-4e46-868b-b77730391a76 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:33.491058 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 167/339] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:33.492159 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad82f72-1d71-4e46-868b-b77730391a76 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:33.613165 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad82f72-1d71-4e46-868b-b77730391a76 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:33.617897 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 173/340] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:33.628526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c62f9-f004-49b7-b8ab-d324471f2d1a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:33.628526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c62f9-f004-49b7-b8ab-d324471f2d1a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:33.628526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c62f9-f004-49b7-b8ab-d324471f2d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:33.653497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119037c-6c9c-46dc-a57a-f03ace09381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:33.654281 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119037c-6c9c-46dc-a57a-f03ace09381f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:33.654567 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119037c-6c9c-46dc-a57a-f03ace09381f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:33.654844 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119037c-6c9c-46dc-a57a-f03ace09381f None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:33.655604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119037c-6c9c-46dc-a57a-f03ace09381f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:33.655900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1c4c62f9-f004-49b7-b8ab-d324471f2d1a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:33.656003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119037c-6c9c-46dc-a57a-f03ace09381f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:33.657385 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119037c-6c9c-46dc-a57a-f03ace09381f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:33.657779 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1c4c62f9-f004-49b7-b8ab-d324471f2d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:33.669691 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 168/341] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:15:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:33.670537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119037c-6c9c-46dc-a57a-f03ace09381f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:33.688380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:33.714019 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:33.737221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 174/342] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:33.738936 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf8304-b112-40f1-bc9b-0637a89d2648 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:33.739527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf8304-b112-40f1-bc9b-0637a89d2648 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:33.740001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf8304-b112-40f1-bc9b-0637a89d2648 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:33.742418 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 175/343] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:33.743786 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf8304-b112-40f1-bc9b-0637a89d2648 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:33.804218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119037c-6c9c-46dc-a57a-f03ace09381f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:33.841627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84fbcd-83e4-459c-965a-5ba3efe82d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:33.842781 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84fbcd-83e4-459c-965a-5ba3efe82d56 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:33.843265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84fbcd-83e4-459c-965a-5ba3efe82d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:33.843737 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84fbcd-83e4-459c-965a-5ba3efe82d56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:33.857024 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd84fbcd-83e4-459c-965a-5ba3efe82d56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:33.866091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-dd84fbcd-83e4-459c-965a-5ba3efe82d56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:33.866091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:33.866091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:33.866091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:33.866091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:33.866091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:33.866091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:33.866091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:33.866091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:33.866091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:33.866091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:33.866091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:33.866091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:33.866091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:33.866091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:33.866091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:33.867027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:33.867027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:33.867027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:33.867027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:33.867027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:33.867027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:33.867027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:33.867027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:33.867027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:33.867027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:33.867027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:33.867027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:33.867027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:33.867027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:33.867027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:33.867027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:33.867833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:33.867833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:33.867833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:33.867833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:33.867833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:33.867833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:33.867833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:33.867833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:33.867833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:33.867833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:33.867833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:33.867833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:33.870526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 169/344] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:33.871956 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aef01d-a007-42f6-a52e-cdbccd54ef40 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:33.872446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aef01d-a007-42f6-a52e-cdbccd54ef40 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:33.872969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aef01d-a007-42f6-a52e-cdbccd54ef40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:33.880947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf8304-b112-40f1-bc9b-0637a89d2648 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:33.900807 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-52aef01d-a007-42f6-a52e-cdbccd54ef40 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:33.902333 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-52aef01d-a007-42f6-a52e-cdbccd54ef40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:33.935102 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf733f-8009-4c59-ab46-a01ff02b52f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:33.936229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf733f-8009-4c59-ab46-a01ff02b52f0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:33.936736 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf733f-8009-4c59-ab46-a01ff02b52f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:33.937158 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf733f-8009-4c59-ab46-a01ff02b52f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:33.985942 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:33.989284 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 170/345] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:33.991046 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aef01d-a007-42f6-a52e-cdbccd54ef40 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:34.015851 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:34.055959 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf733f-8009-4c59-ab46-a01ff02b52f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:34.056299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf733f-8009-4c59-ab46-a01ff02b52f0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:34.143728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aef01d-a007-42f6-a52e-cdbccd54ef40 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:34.197696 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c5203-2473-4240-8d32-db037cec7e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:34.198856 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c5203-2473-4240-8d32-db037cec7e9b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:34.199052 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c5203-2473-4240-8d32-db037cec7e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:34.199340 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c5203-2473-4240-8d32-db037cec7e9b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:34.210758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 176/346] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:34.212206 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf733f-8009-4c59-ab46-a01ff02b52f0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:34.224625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c5203-2473-4240-8d32-db037cec7e9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:34.235216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c74c5203-2473-4240-8d32-db037cec7e9b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:15:34.235216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:34.235216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:34.235216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:34.235216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:34.235216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:34.235216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:34.235216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:34.235216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:34.235216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:34.235216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:34.235216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:34.235216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:34.235216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:34.235216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:34.235216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:34.236131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:34.236131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:34.236131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:34.236131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:34.236131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:34.236131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:34.236131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:34.236131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:34.236131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:34.236131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:34.236131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:34.236131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:34.236131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:34.236131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:34.236131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:34.236131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:34.237014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:34.237014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:34.237014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:34.237014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:34.237014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:34.237014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:34.237014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:34.237014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:34.237014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:34.237014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:34.237014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 10:15:34.237014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:34.237014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 171/347] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:34.237674 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c5203-2473-4240-8d32-db037cec7e9b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:34.287019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:34.343072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf733f-8009-4c59-ab46-a01ff02b52f0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:34.344251 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:34.371956 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c5203-2473-4240-8d32-db037cec7e9b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:34.391996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee869b89-3dc0-4948-ac29-9af24f2e5cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:34.393180 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee869b89-3dc0-4948-ac29-9af24f2e5cfb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:34.394112 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee869b89-3dc0-4948-ac29-9af24f2e5cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:34.394620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee869b89-3dc0-4948-ac29-9af24f2e5cfb None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:34.395866 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee869b89-3dc0-4948-ac29-9af24f2e5cfb None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:34.396374 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee869b89-3dc0-4948-ac29-9af24f2e5cfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:34.398727 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee869b89-3dc0-4948-ac29-9af24f2e5cfb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:34.413891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 177/348] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:34.414887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee869b89-3dc0-4948-ac29-9af24f2e5cfb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:34.424943 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02213d-9728-4213-a5ea-6ac257926b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:34.426066 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02213d-9728-4213-a5ea-6ac257926b37 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:34.426548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02213d-9728-4213-a5ea-6ac257926b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:34.427022 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02213d-9728-4213-a5ea-6ac257926b37 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:34.443973 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02213d-9728-4213-a5ea-6ac257926b37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:34.455255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4d02213d-9728-4213-a5ea-6ac257926b37 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:34.455255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:34.455255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:34.455255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:34.455255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:34.455255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:34.455255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:34.455255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:34.455255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:34.455255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:34.455255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:34.455255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:34.455255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:34.455255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:34.455255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:34.455255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:34.456116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:34.456116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:34.456116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:34.456116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:34.456116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:34.456116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:34.456116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:34.456116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:34.456116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:34.456116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:34.456116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:34.456116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:34.456116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:34.456116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:34.456116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:34.456116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:34.456967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:34.456967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:34.456967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:34.456967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:34.456967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:34.456967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:34.456967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:34.456967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:34.456967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:34.456967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:34.456967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:34.456967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:34.460449 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 172/349] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:34.465459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02213d-9728-4213-a5ea-6ac257926b37 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:34.555459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee869b89-3dc0-4948-ac29-9af24f2e5cfb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:34.586850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02213d-9728-4213-a5ea-6ac257926b37 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:34.587490 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:34.604142 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20de39-0fa0-4025-9eb2-1d4f8d64c021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:34.605609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20de39-0fa0-4025-9eb2-1d4f8d64c021 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:34.606123 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20de39-0fa0-4025-9eb2-1d4f8d64c021 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:34.607054 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20de39-0fa0-4025-9eb2-1d4f8d64c021 None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:34.608703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20de39-0fa0-4025-9eb2-1d4f8d64c021 None admin] RBAC: Authorizing `identity:get_project(project_id=09d730b36d8f4ab482dc9ed98757bb0b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:34.611019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20de39-0fa0-4025-9eb2-1d4f8d64c021 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:34.613796 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 178/350] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:34 2022] GET /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:34.614820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20de39-0fa0-4025-9eb2-1d4f8d64c021 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:34.629358 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6ab25-6b7a-4889-bfc1-83fe9d67f952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:34.630034 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6ab25-6b7a-4889-bfc1-83fe9d67f952 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:34.630675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6ab25-6b7a-4889-bfc1-83fe9d67f952 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:34.630675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6ab25-6b7a-4889-bfc1-83fe9d67f952 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:34.631456 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6ab25-6b7a-4889-bfc1-83fe9d67f952 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:34.631878 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6ab25-6b7a-4889-bfc1-83fe9d67f952 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:34.634559 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6ab25-6b7a-4889-bfc1-83fe9d67f952 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:34.647687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 173/351] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:34.648492 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6ab25-6b7a-4889-bfc1-83fe9d67f952 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:34.655339 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:34.759145 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20de39-0fa0-4025-9eb2-1d4f8d64c021 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:34.796123 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6ab25-6b7a-4889-bfc1-83fe9d67f952 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:34.799431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a596a-96de-4c57-8aa3-76f919991c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:34.799992 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a596a-96de-4c57-8aa3-76f919991c66 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:34.800280 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a596a-96de-4c57-8aa3-76f919991c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:34.800587 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a596a-96de-4c57-8aa3-76f919991c66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:34.839975 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca5af2-2b97-4267-bfcd-4bdda5b6e975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:34.841285 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca5af2-2b97-4267-bfcd-4bdda5b6e975 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:34.841775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca5af2-2b97-4267-bfcd-4bdda5b6e975 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:34.843082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca5af2-2b97-4267-bfcd-4bdda5b6e975 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:34.844059 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca5af2-2b97-4267-bfcd-4bdda5b6e975 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:34.844595 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca5af2-2b97-4267-bfcd-4bdda5b6e975 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:34.846193 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca5af2-2b97-4267-bfcd-4bdda5b6e975 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:34.858901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 174/352] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:34.860052 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca5af2-2b97-4267-bfcd-4bdda5b6e975 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:34.875683 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:34.890629 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a596a-96de-4c57-8aa3-76f919991c66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:34.892943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a596a-96de-4c57-8aa3-76f919991c66 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:34.950014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:35.001481 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca5af2-2b97-4267-bfcd-4bdda5b6e975 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:35.023586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 179/353] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:35.025119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a596a-96de-4c57-8aa3-76f919991c66 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:35.046916 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d205a2-ee5b-4d53-bdea-2ca7fcff7a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:35.047781 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d205a2-ee5b-4d53-bdea-2ca7fcff7a63 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:35.048064 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d205a2-ee5b-4d53-bdea-2ca7fcff7a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:35.048342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d205a2-ee5b-4d53-bdea-2ca7fcff7a63 None admin] PATH_INFO: `/v3/groups/cd82bf61901242f29b02839bb7fb7f51` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:35.056596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d205a2-ee5b-4d53-bdea-2ca7fcff7a63 None admin] RBAC: Authorizing `identity:get_group(group_id=cd82bf61901242f29b02839bb7fb7f51)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:35.058075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d205a2-ee5b-4d53-bdea-2ca7fcff7a63 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:35.060843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 175/354] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 18 10:15:34 2022] GET /identity/v3/groups/cd82bf61901242f29b02839bb7fb7f51 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:35.061627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d205a2-ee5b-4d53-bdea-2ca7fcff7a63 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:35.136757 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:35.151052 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a596a-96de-4c57-8aa3-76f919991c66 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:35.199419 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf492ba-a047-429d-9e33-53adfd904aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:35.200315 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf492ba-a047-429d-9e33-53adfd904aff None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:35.200633 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf492ba-a047-429d-9e33-53adfd904aff None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:35.200922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf492ba-a047-429d-9e33-53adfd904aff None admin] PATH_INFO: `/v3/users/2b246182e6a649fc9d2439e9d1badc66` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:35.202347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d205a2-ee5b-4d53-bdea-2ca7fcff7a63 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:35.204379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf492ba-a047-429d-9e33-53adfd904aff None admin] RBAC: Authorizing `identity:get_user(user_id=2b246182e6a649fc9d2439e9d1badc66)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:35.206021 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf492ba-a047-429d-9e33-53adfd904aff None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:35.207861 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 180/355] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:15:35 2022] GET /identity/v3/users/2b246182e6a649fc9d2439e9d1badc66 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:35.208984 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a3edc-81ae-4c94-9ec8-7a9dce0dfcbf None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:35.209271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a3edc-81ae-4c94-9ec8-7a9dce0dfcbf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:35.209550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a3edc-81ae-4c94-9ec8-7a9dce0dfcbf None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:35.211968 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 181/356] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:35.211968 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a3edc-81ae-4c94-9ec8-7a9dce0dfcbf None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:35.239485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:35.249010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11bfb99-b8cf-4c6f-ab05-c2ca100b5331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:35.249778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bfb99-b8cf-4c6f-ab05-c2ca100b5331 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:35.250147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bfb99-b8cf-4c6f-ab05-c2ca100b5331 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:35.250349 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11bfb99-b8cf-4c6f-ab05-c2ca100b5331 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:35.337624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11bfb99-b8cf-4c6f-ab05-c2ca100b5331 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:35.339045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11bfb99-b8cf-4c6f-ab05-c2ca100b5331 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:35.346403 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a3edc-81ae-4c94-9ec8-7a9dce0dfcbf None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:35.388956 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c41e8-d368-43c1-8e5f-7e9d2a1b29df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:35.390168 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c41e8-d368-43c1-8e5f-7e9d2a1b29df None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:35.390449 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c41e8-d368-43c1-8e5f-7e9d2a1b29df None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:35.390726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c41e8-d368-43c1-8e5f-7e9d2a1b29df None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:35.391922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c41e8-d368-43c1-8e5f-7e9d2a1b29df None admin] RBAC: Authorizing `identity:get_project(project_id=09d730b36d8f4ab482dc9ed98757bb0b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:35.393443 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c41e8-d368-43c1-8e5f-7e9d2a1b29df None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:35.395147 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 182/357] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:35 2022] GET /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:35.395915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c41e8-d368-43c1-8e5f-7e9d2a1b29df None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:35.426516 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:35.463370 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 176/358] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:35.464231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11bfb99-b8cf-4c6f-ab05-c2ca100b5331 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:35.500284 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:35.505524 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c41e8-d368-43c1-8e5f-7e9d2a1b29df None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:35.544027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f7e13-aa56-4a01-86ff-6d8f7ef03070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:35.544717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f7e13-aa56-4a01-86ff-6d8f7ef03070 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:35.545022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f7e13-aa56-4a01-86ff-6d8f7ef03070 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:35.545275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f7e13-aa56-4a01-86ff-6d8f7ef03070 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:35.546120 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f7e13-aa56-4a01-86ff-6d8f7ef03070 None admin] RBAC: Adding query filter params (group.id=cd82bf61901242f29b02839bb7fb7f51, role.id=8c039234db9640d0bdc0c2a629ff9e52, scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:35.546432 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f7e13-aa56-4a01-86ff-6d8f7ef03070 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:35.547873 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f7e13-aa56-4a01-86ff-6d8f7ef03070 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:35.555209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 183/359] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 18 10:15:35 2022] GET /identity/v3/role_assignments?group.id=cd82bf61901242f29b02839bb7fb7f51&scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b&role.id=8c039234db9640d0bdc0c2a629ff9e52 => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:35.556440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90117c-47e7-44bd-a0e0-adfb0918d32a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:35.556762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90117c-47e7-44bd-a0e0-adfb0918d32a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:35.557041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90117c-47e7-44bd-a0e0-adfb0918d32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:35.577872 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ba90117c-47e7-44bd-a0e0-adfb0918d32a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:35.579340 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ba90117c-47e7-44bd-a0e0-adfb0918d32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:35.602522 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11bfb99-b8cf-4c6f-ab05-c2ca100b5331 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:35.654352 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717fbc0-e546-4ebf-af24-33e0ea285e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:35.655021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717fbc0-e546-4ebf-af24-33e0ea285e3b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:35.655306 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717fbc0-e546-4ebf-af24-33e0ea285e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:35.655580 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717fbc0-e546-4ebf-af24-33e0ea285e3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:35.656426 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717fbc0-e546-4ebf-af24-33e0ea285e3b None admin] RBAC: Adding query filter params (role.id=e6c6a363f7034bd0a72b72ffe324aab8, user.id=2b246182e6a649fc9d2439e9d1badc66, scope.system=all) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:35.656771 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717fbc0-e546-4ebf-af24-33e0ea285e3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:35.658348 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717fbc0-e546-4ebf-af24-33e0ea285e3b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:35.672531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 184/360] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:35.674044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d229a6c-cff7-4dda-8de9-49b78ac0558e None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:35.674801 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d229a6c-cff7-4dda-8de9-49b78ac0558e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:35.674801 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d229a6c-cff7-4dda-8de9-49b78ac0558e None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:35.676288 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 185/361] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:35.677174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d229a6c-cff7-4dda-8de9-49b78ac0558e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:35.684666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 177/362] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Jul 18 10:15:35 2022] GET /identity/v3/role_assignments?user.id=2b246182e6a649fc9d2439e9d1badc66&scope.system=all&role.id=e6c6a363f7034bd0a72b72ffe324aab8 => generated 510 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:35.685823 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717fbc0-e546-4ebf-af24-33e0ea285e3b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:35.708151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:35.819095 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d229a6c-cff7-4dda-8de9-49b78ac0558e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:35.839624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:35.860869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717fbc0-e546-4ebf-af24-33e0ea285e3b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:35.875923 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3289a2f-5773-485b-a00e-311b83646419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:35.876825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3289a2f-5773-485b-a00e-311b83646419 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:35.877112 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3289a2f-5773-485b-a00e-311b83646419 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:35.877378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3289a2f-5773-485b-a00e-311b83646419 None admin] PATH_INFO: `/v3/groups/cd82bf61901242f29b02839bb7fb7f51` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:35.878587 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3289a2f-5773-485b-a00e-311b83646419 None admin] RBAC: Authorizing `identity:get_group(group_id=cd82bf61901242f29b02839bb7fb7f51)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:35.880097 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3289a2f-5773-485b-a00e-311b83646419 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:35.886133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 186/363] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 18 10:15:35 2022] GET /identity/v3/groups/cd82bf61901242f29b02839bb7fb7f51 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:35.886133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a665b1-0f4e-42b4-8608-44129586fa2d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:35.886133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a665b1-0f4e-42b4-8608-44129586fa2d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:35.886133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a665b1-0f4e-42b4-8608-44129586fa2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:35.904235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-97a665b1-0f4e-42b4-8608-44129586fa2d None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:35.905478 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-97a665b1-0f4e-42b4-8608-44129586fa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:35.917623 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86eb21b-d282-4130-9313-66ab62b992e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:35.918855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86eb21b-d282-4130-9313-66ab62b992e0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:35.919143 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86eb21b-d282-4130-9313-66ab62b992e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:35.919412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86eb21b-d282-4130-9313-66ab62b992e0 None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:35.921232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86eb21b-d282-4130-9313-66ab62b992e0 None admin] RBAC: Authorizing `identity:get_project(project_id=57b2ea019ba3413bafb7714839f85a22)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:35.923216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86eb21b-d282-4130-9313-66ab62b992e0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:35.925582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 178/364] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:35 2022] GET /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:35.926532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86eb21b-d282-4130-9313-66ab62b992e0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:35.995878 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 187/365] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:35.996598 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a665b1-0f4e-42b4-8608-44129586fa2d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:36.022147 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:36.053348 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86eb21b-d282-4130-9313-66ab62b992e0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:36.096117 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2077045-a829-4fc0-a0c9-15ec0de91ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:36.097013 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2077045-a829-4fc0-a0c9-15ec0de91ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:36.097299 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2077045-a829-4fc0-a0c9-15ec0de91ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:36.097586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2077045-a829-4fc0-a0c9-15ec0de91ad2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:36.114917 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2077045-a829-4fc0-a0c9-15ec0de91ad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:36.126076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c2077045-a829-4fc0-a0c9-15ec0de91ad2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:36.126076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:36.126076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:36.126076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:36.126076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:36.126076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:36.126076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:36.126076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:36.126076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:36.126076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:36.126076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:36.126076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:36.126076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:36.126076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:36.126076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:36.126076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:36.127011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:36.127011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:36.127011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:36.127011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:36.127011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:36.127011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:36.127011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:36.127011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:36.127011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:36.127011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:36.127011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:36.127011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:36.127011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:36.127011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:36.127011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:36.127011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:36.127810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:36.127810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:36.127810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:36.127810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:36.127810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:36.127810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:36.127810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:36.127810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:36.127810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:36.127810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:36.127810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:36.127810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:36.127810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 179/366] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:15:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:36.128474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a665b1-0f4e-42b4-8608-44129586fa2d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:36.129140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2077045-a829-4fc0-a0c9-15ec0de91ad2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:36.149529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:36.168443 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79730fab-72d7-43be-93b9-3bb75c552229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:36.169160 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79730fab-72d7-43be-93b9-3bb75c552229 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:36.169451 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79730fab-72d7-43be-93b9-3bb75c552229 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:36.169739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79730fab-72d7-43be-93b9-3bb75c552229 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:36.170696 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79730fab-72d7-43be-93b9-3bb75c552229 None admin] RBAC: Adding query filter params (group.id=cd82bf61901242f29b02839bb7fb7f51, role.id=2820090979cb487db491f835e03e6471, scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:36.171020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79730fab-72d7-43be-93b9-3bb75c552229 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:36.173595 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79730fab-72d7-43be-93b9-3bb75c552229 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:36.183580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 188/367] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 18 10:15:35 2022] GET /identity/v3/role_assignments?group.id=cd82bf61901242f29b02839bb7fb7f51&scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b&role.id=2820090979cb487db491f835e03e6471 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:36.183580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79730fab-72d7-43be-93b9-3bb75c552229 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:36.259492 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2077045-a829-4fc0-a0c9-15ec0de91ad2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:36.312075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2377f-b43c-4687-bac2-5faf60c2adfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:36.313875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2377f-b43c-4687-bac2-5faf60c2adfd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:36.314835 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2377f-b43c-4687-bac2-5faf60c2adfd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:36.315480 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2377f-b43c-4687-bac2-5faf60c2adfd None admin] PATH_INFO: `/v3/groups/cd82bf61901242f29b02839bb7fb7f51` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:36.317371 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2377f-b43c-4687-bac2-5faf60c2adfd None admin] RBAC: Authorizing `identity:get_group(group_id=cd82bf61901242f29b02839bb7fb7f51)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:36.319759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2377f-b43c-4687-bac2-5faf60c2adfd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:36.321784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:36.324618 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 180/368] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 18 10:15:36 2022] GET /identity/v3/groups/cd82bf61901242f29b02839bb7fb7f51 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:36.325717 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2377f-b43c-4687-bac2-5faf60c2adfd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:36.348436 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79730fab-72d7-43be-93b9-3bb75c552229 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:36.406402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e293f2-6929-457c-9bc8-36d1f328c499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:36.407223 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e293f2-6929-457c-9bc8-36d1f328c499 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:36.407578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e293f2-6929-457c-9bc8-36d1f328c499 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:36.407923 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e293f2-6929-457c-9bc8-36d1f328c499 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:36.421692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e293f2-6929-457c-9bc8-36d1f328c499 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:36.433832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-91e293f2-6929-457c-9bc8-36d1f328c499 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:15:36.433832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:36.433832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:36.433832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:36.433832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:36.433832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:36.433832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:36.433832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:36.433832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:36.433832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:36.433832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:36.433832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:36.433832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:36.433832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:36.433832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:36.433832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:36.434916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:36.434916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:36.434916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:36.434916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:36.434916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:36.434916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:36.434916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:36.434916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:36.434916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:36.434916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:36.434916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:36.434916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:36.434916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:36.434916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:36.434916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:36.434916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:36.435783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:36.435783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:36.435783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:36.435783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:36.435783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:36.435783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:36.435783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:36.435783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:36.435783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:36.435783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:36.435783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:15:36.435783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:36.435783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 189/369] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:15:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:36.436486 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e293f2-6929-457c-9bc8-36d1f328c499 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:36.450292 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:36.474276 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2377f-b43c-4687-bac2-5faf60c2adfd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:36.513945 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2ad3a-ff68-412f-920b-5bc777ffd40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:36.514497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2ad3a-ff68-412f-920b-5bc777ffd40d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:36.514854 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2ad3a-ff68-412f-920b-5bc777ffd40d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:36.515267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2ad3a-ff68-412f-920b-5bc777ffd40d None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:36.517058 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2ad3a-ff68-412f-920b-5bc777ffd40d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:36.517442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2ad3a-ff68-412f-920b-5bc777ffd40d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:36.518898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2ad3a-ff68-412f-920b-5bc777ffd40d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:36.529049 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 181/370] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:15:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:36.529798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2ad3a-ff68-412f-920b-5bc777ffd40d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:36.563823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e293f2-6929-457c-9bc8-36d1f328c499 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:36.600656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a64f2-a73e-4cf9-81c8-a4d2af9c8abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:36.601653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a64f2-a73e-4cf9-81c8-a4d2af9c8abe None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:36.602128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a64f2-a73e-4cf9-81c8-a4d2af9c8abe None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:36.602589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a64f2-a73e-4cf9-81c8-a4d2af9c8abe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:36.603615 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a64f2-a73e-4cf9-81c8-a4d2af9c8abe None admin] RBAC: Adding query filter params (group.id=cd82bf61901242f29b02839bb7fb7f51, role.id=2820090979cb487db491f835e03e6471, scope.project.id=57b2ea019ba3413bafb7714839f85a22) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:36.604409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a64f2-a73e-4cf9-81c8-a4d2af9c8abe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:36.606109 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a64f2-a73e-4cf9-81c8-a4d2af9c8abe None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:36.612239 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 190/371] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 18 10:15:36 2022] GET /identity/v3/role_assignments?group.id=cd82bf61901242f29b02839bb7fb7f51&scope.project.id=57b2ea019ba3413bafb7714839f85a22&role.id=2820090979cb487db491f835e03e6471 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:36.613319 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a64f2-a73e-4cf9-81c8-a4d2af9c8abe None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:36.634928 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:36.653517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2ad3a-ff68-412f-920b-5bc777ffd40d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:36.776919 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a64f2-a73e-4cf9-81c8-a4d2af9c8abe None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:36.781269 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c092ed-eabe-45db-be0c-4468d023e29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:36.781967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c092ed-eabe-45db-be0c-4468d023e29e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:36.782258 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c092ed-eabe-45db-be0c-4468d023e29e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:36.782539 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c092ed-eabe-45db-be0c-4468d023e29e None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:36.783313 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c092ed-eabe-45db-be0c-4468d023e29e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:36.783628 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c092ed-eabe-45db-be0c-4468d023e29e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:36.785067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c092ed-eabe-45db-be0c-4468d023e29e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:36.808312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 182/372] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:36.820616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c092ed-eabe-45db-be0c-4468d023e29e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:36.824178 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ffbda-aefa-4432-84cd-5ca75ad7cf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:36.824891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ffbda-aefa-4432-84cd-5ca75ad7cf45 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:36.825179 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ffbda-aefa-4432-84cd-5ca75ad7cf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:36.825458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ffbda-aefa-4432-84cd-5ca75ad7cf45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:36.834024 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:36.917378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ffbda-aefa-4432-84cd-5ca75ad7cf45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:36.918879 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ffbda-aefa-4432-84cd-5ca75ad7cf45 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:36.940926 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:36.964981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c092ed-eabe-45db-be0c-4468d023e29e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:37.003486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb609a-715d-4d68-80c4-98479dd62c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:37.004596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb609a-715d-4d68-80c4-98479dd62c9c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:37.005105 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb609a-715d-4d68-80c4-98479dd62c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:37.005586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb609a-715d-4d68-80c4-98479dd62c9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:37.092700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb609a-715d-4d68-80c4-98479dd62c9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:37.094517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb609a-715d-4d68-80c4-98479dd62c9c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:37.104138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 191/373] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:37.110653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ffbda-aefa-4432-84cd-5ca75ad7cf45 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:37.128912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:37.251187 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 183/374] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:37.253730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ffbda-aefa-4432-84cd-5ca75ad7cf45 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:37.257607 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb609a-715d-4d68-80c4-98479dd62c9c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:37.265640 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:37.296820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f79eb-e6ba-414e-877c-76f607b36e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:37.297856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f79eb-e6ba-414e-877c-76f607b36e0c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:37.298286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f79eb-e6ba-414e-877c-76f607b36e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:37.298569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f79eb-e6ba-414e-877c-76f607b36e0c None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:37.303005 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f79eb-e6ba-414e-877c-76f607b36e0c None admin] RBAC: Authorizing `identity:get_project(project_id=57b2ea019ba3413bafb7714839f85a22)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:37.305298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f79eb-e6ba-414e-877c-76f607b36e0c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:37.308652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 192/375] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:37 2022] GET /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:37.314208 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f79eb-e6ba-414e-877c-76f607b36e0c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:37.440850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb609a-715d-4d68-80c4-98479dd62c9c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:37.457295 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:37.487485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2445d7e-e3c0-490f-86bf-ffe3bc27309a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:37.488724 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2445d7e-e3c0-490f-86bf-ffe3bc27309a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:37.489270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2445d7e-e3c0-490f-86bf-ffe3bc27309a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:37.489724 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2445d7e-e3c0-490f-86bf-ffe3bc27309a None admin] PATH_INFO: `/v3/projects/1282af50d400435ba3f0e00de6e5fb67` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:37.491279 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2445d7e-e3c0-490f-86bf-ffe3bc27309a None admin] RBAC: Authorizing `identity:get_project(project_id=1282af50d400435ba3f0e00de6e5fb67)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:37.494034 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2445d7e-e3c0-490f-86bf-ffe3bc27309a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:37.496408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 184/376] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:37 2022] GET /identity/v3/projects/1282af50d400435ba3f0e00de6e5fb67 => generated 344 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:37.505662 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2445d7e-e3c0-490f-86bf-ffe3bc27309a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:37.535784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f79eb-e6ba-414e-877c-76f607b36e0c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:37.642474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05f0e9-4679-4a1d-8665-30508b531205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:37.643404 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05f0e9-4679-4a1d-8665-30508b531205 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:37.643690 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05f0e9-4679-4a1d-8665-30508b531205 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:37.643967 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05f0e9-4679-4a1d-8665-30508b531205 None admin] PATH_INFO: `/v3/groups/cd82bf61901242f29b02839bb7fb7f51` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:37.645609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05f0e9-4679-4a1d-8665-30508b531205 None admin] RBAC: Authorizing `identity:get_group(group_id=cd82bf61901242f29b02839bb7fb7f51)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:37.647229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05f0e9-4679-4a1d-8665-30508b531205 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:37.649600 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 193/377] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 18 10:15:37 2022] GET /identity/v3/groups/cd82bf61901242f29b02839bb7fb7f51 => generated 226 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:37.655735 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05f0e9-4679-4a1d-8665-30508b531205 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:37.697992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2445d7e-e3c0-490f-86bf-ffe3bc27309a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:37.760548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:37.776935 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce7232-1ba7-4c00-b696-8b4bf2b81a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:37.778948 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce7232-1ba7-4c00-b696-8b4bf2b81a16 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:37.779247 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce7232-1ba7-4c00-b696-8b4bf2b81a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:37.779529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce7232-1ba7-4c00-b696-8b4bf2b81a16 None admin] PATH_INFO: `/v3/groups/d4f3ac68087d4d788393b4b51a0d096c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:37.792958 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce7232-1ba7-4c00-b696-8b4bf2b81a16 None admin] RBAC: Authorizing `identity:get_group(group_id=d4f3ac68087d4d788393b4b51a0d096c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:37.794431 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce7232-1ba7-4c00-b696-8b4bf2b81a16 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:37.798360 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 185/378] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 18 10:15:37 2022] GET /identity/v3/groups/d4f3ac68087d4d788393b4b51a0d096c => generated 229 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:37.804557 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce7232-1ba7-4c00-b696-8b4bf2b81a16 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:37.884452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05f0e9-4679-4a1d-8665-30508b531205 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:37.896877 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:37.956123 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f34232-8f51-4fe1-8fab-7d60347942ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:37.956746 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f34232-8f51-4fe1-8fab-7d60347942ab None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:37.957029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f34232-8f51-4fe1-8fab-7d60347942ab None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:37.957302 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f34232-8f51-4fe1-8fab-7d60347942ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:37.958149 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f34232-8f51-4fe1-8fab-7d60347942ab None admin] RBAC: Adding query filter params (group.id=cd82bf61901242f29b02839bb7fb7f51, role.id=8c039234db9640d0bdc0c2a629ff9e52, scope.project.id=57b2ea019ba3413bafb7714839f85a22) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:37.958459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f34232-8f51-4fe1-8fab-7d60347942ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:37.960064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f34232-8f51-4fe1-8fab-7d60347942ab None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:37.978679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 194/379] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 18 10:15:37 2022] GET /identity/v3/role_assignments?group.id=cd82bf61901242f29b02839bb7fb7f51&scope.project.id=57b2ea019ba3413bafb7714839f85a22&role.id=8c039234db9640d0bdc0c2a629ff9e52 => generated 260 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:38.012219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce7232-1ba7-4c00-b696-8b4bf2b81a16 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:38.055093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb139d0-f76a-45b9-9b31-9d359f58bc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:38.055782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb139d0-f76a-45b9-9b31-9d359f58bc6c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:38.056074 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb139d0-f76a-45b9-9b31-9d359f58bc6c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:38.056360 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb139d0-f76a-45b9-9b31-9d359f58bc6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:38.057260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb139d0-f76a-45b9-9b31-9d359f58bc6c None admin] RBAC: Adding query filter params (group.id=d4f3ac68087d4d788393b4b51a0d096c, role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.project.id=1282af50d400435ba3f0e00de6e5fb67) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:38.057581 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb139d0-f76a-45b9-9b31-9d359f58bc6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:38.059061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb139d0-f76a-45b9-9b31-9d359f58bc6c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:38.065678 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 186/380] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 18 10:15:37 2022] GET /identity/v3/role_assignments?group.id=d4f3ac68087d4d788393b4b51a0d096c&scope.project.id=1282af50d400435ba3f0e00de6e5fb67&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 260 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:38.110292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30f460-7e53-4167-adc8-0c53515bc31f None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:38.110760 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30f460-7e53-4167-adc8-0c53515bc31f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:38.111231 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30f460-7e53-4167-adc8-0c53515bc31f None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:38.115086 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 195/381] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:38.120158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c477c-7bfc-45d7-9ccb-7dc15e3eb07a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:38.120580 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c477c-7bfc-45d7-9ccb-7dc15e3eb07a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:38.120834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c477c-7bfc-45d7-9ccb-7dc15e3eb07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:38.141023 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-155c477c-7bfc-45d7-9ccb-7dc15e3eb07a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:38.142256 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-155c477c-7bfc-45d7-9ccb-7dc15e3eb07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:38.247262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 187/382] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:38.273286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30f460-7e53-4167-adc8-0c53515bc31f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:38.427496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30f460-7e53-4167-adc8-0c53515bc31f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:38.438279 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:38.476065 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf8e55-633d-40bf-8d0f-2beef8972df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:38.476916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf8e55-633d-40bf-8d0f-2beef8972df8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:38.477202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf8e55-633d-40bf-8d0f-2beef8972df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:38.477479 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf8e55-633d-40bf-8d0f-2beef8972df8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:38.494316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf8e55-633d-40bf-8d0f-2beef8972df8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:38.502567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b6bf8e55-633d-40bf-8d0f-2beef8972df8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:38.502567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:38.502567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:38.502567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:38.502567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:38.502567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:38.502567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:38.502567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:38.502567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:38.502567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:38.502567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:38.502567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:38.502567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:38.502567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:38.502567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:38.502567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:38.504568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:38.504568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:38.504568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:38.504568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:38.504568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:38.504568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:38.504568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:38.504568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:38.504568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:38.504568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:38.504568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:38.504568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:38.504568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:38.504568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:38.504568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:38.504568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:38.506275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:38.506275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:38.506275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:38.506275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:38.506275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:38.506275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:38.506275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:38.506275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:38.506275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:38.506275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:38.506275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:38.506275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:38.506275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 196/383] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:15:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:38.512084 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c477c-7bfc-45d7-9ccb-7dc15e3eb07a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:38.612852 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:38.675323 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c477c-7bfc-45d7-9ccb-7dc15e3eb07a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:38.722364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434687d3-a3d2-4c64-a05b-9d4f790eb74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:38.723100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434687d3-a3d2-4c64-a05b-9d4f790eb74f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:38.723406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434687d3-a3d2-4c64-a05b-9d4f790eb74f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:38.723793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434687d3-a3d2-4c64-a05b-9d4f790eb74f None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:38.724555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434687d3-a3d2-4c64-a05b-9d4f790eb74f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:38.724842 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434687d3-a3d2-4c64-a05b-9d4f790eb74f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:38.726771 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434687d3-a3d2-4c64-a05b-9d4f790eb74f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:38.739892 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 188/384] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:15:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:38.746446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf8e55-633d-40bf-8d0f-2beef8972df8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:38.869862 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1b9a1-d5e9-49dc-be9e-6fea1169cb10 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:38.870286 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1b9a1-d5e9-49dc-be9e-6fea1169cb10 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:38.870444 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1b9a1-d5e9-49dc-be9e-6fea1169cb10 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:38.872143 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 189/385] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:38.877523 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed28b86-27bf-40b9-9d42-b6765fdff2af None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:38.877812 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed28b86-27bf-40b9-9d42-b6765fdff2af None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:38.878092 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed28b86-27bf-40b9-9d42-b6765fdff2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:38.896756 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6ed28b86-27bf-40b9-9d42-b6765fdff2af None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:38.898675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6ed28b86-27bf-40b9-9d42-b6765fdff2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:38.943234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf8e55-633d-40bf-8d0f-2beef8972df8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:38.992478 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f478c-f261-4919-b6d6-ab4ae18594d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:38.993369 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f478c-f261-4919-b6d6-ab4ae18594d9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:38.993652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f478c-f261-4919-b6d6-ab4ae18594d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:38.993926 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f478c-f261-4919-b6d6-ab4ae18594d9 None admin] PATH_INFO: `/v3/groups/cd82bf61901242f29b02839bb7fb7f51` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:38.996121 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f478c-f261-4919-b6d6-ab4ae18594d9 None admin] RBAC: Authorizing `identity:get_group(group_id=cd82bf61901242f29b02839bb7fb7f51)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:38.997609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f478c-f261-4919-b6d6-ab4ae18594d9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:38.999249 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 197/386] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 18 10:15:38 2022] GET /identity/v3/groups/cd82bf61901242f29b02839bb7fb7f51 => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:39.005573 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f478c-f261-4919-b6d6-ab4ae18594d9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:39.029562 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 190/387] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:39.053343 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:39.070962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed28b86-27bf-40b9-9d42-b6765fdff2af None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:39.161116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:39.221583 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f478c-f261-4919-b6d6-ab4ae18594d9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:39.325789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646b163-e4ea-4d01-a52b-c1ccadb6f38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:39.327223 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646b163-e4ea-4d01-a52b-c1ccadb6f38c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:39.327717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646b163-e4ea-4d01-a52b-c1ccadb6f38c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:39.328204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646b163-e4ea-4d01-a52b-c1ccadb6f38c None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:39.330194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646b163-e4ea-4d01-a52b-c1ccadb6f38c None admin] RBAC: Authorizing `identity:get_project(project_id=09d730b36d8f4ab482dc9ed98757bb0b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:39.332784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646b163-e4ea-4d01-a52b-c1ccadb6f38c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:39.339583 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 198/388] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:39 2022] GET /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b => generated 298 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:39.346202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646b163-e4ea-4d01-a52b-c1ccadb6f38c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:39.381949 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed28b86-27bf-40b9-9d42-b6765fdff2af None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:39.451668 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954d93e-eae8-421c-a10d-60a61e7bcf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:39.453037 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954d93e-eae8-421c-a10d-60a61e7bcf90 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:39.453519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954d93e-eae8-421c-a10d-60a61e7bcf90 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:39.454001 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954d93e-eae8-421c-a10d-60a61e7bcf90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:39.479559 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e954d93e-eae8-421c-a10d-60a61e7bcf90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:39.513566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e954d93e-eae8-421c-a10d-60a61e7bcf90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:39.513566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:39.513566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:39.513566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:39.513566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:39.513566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:39.513566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:39.513566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:39.513566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:39.513566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:39.513566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:39.513566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:39.513566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:39.513566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:39.513566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:39.513566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:39.516375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:39.516375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:39.516375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:39.516375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:39.516375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:39.516375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:39.516375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:39.516375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:39.516375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:39.516375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:39.516375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:39.516375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:39.516375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:39.516375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:39.516375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:39.516375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:39.518265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:39.518265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:39.518265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:39.518265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:39.518265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:39.518265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:39.518265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:39.518265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:39.518265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:39.518265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:39.518265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:39.518265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:39.518265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 191/389] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 447 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:39.524530 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954d93e-eae8-421c-a10d-60a61e7bcf90 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:39.630052 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646b163-e4ea-4d01-a52b-c1ccadb6f38c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:39.634635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:39.699500 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:39.701019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951986a-ab96-40ab-bcfa-0438a445367a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:39.701950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951986a-ab96-40ab-bcfa-0438a445367a None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:39.702180 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951986a-ab96-40ab-bcfa-0438a445367a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:39.702452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951986a-ab96-40ab-bcfa-0438a445367a None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b/groups/cd82bf61901242f29b02839bb7fb7f51/roles/8c039234db9640d0bdc0c2a629ff9e52` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:39.705826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951986a-ab96-40ab-bcfa-0438a445367a None admin] RBAC: Authorizing `identity:create_grant(project_id=09d730b36d8f4ab482dc9ed98757bb0b, group_id=cd82bf61901242f29b02839bb7fb7f51, role_id=8c039234db9640d0bdc0c2a629ff9e52)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:39.707291 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951986a-ab96-40ab-bcfa-0438a445367a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:39.722679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954d93e-eae8-421c-a10d-60a61e7bcf90 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:39.735404 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 199/390] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Jul 18 10:15:39 2022] PUT /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b/groups/cd82bf61901242f29b02839bb7fb7f51/roles/8c039234db9640d0bdc0c2a629ff9e52 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:15:39.764339 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37ebd0-d666-46e2-9270-4d26c1d11167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:39.765169 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37ebd0-d666-46e2-9270-4d26c1d11167 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:39.765458 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37ebd0-d666-46e2-9270-4d26c1d11167 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:39.765781 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37ebd0-d666-46e2-9270-4d26c1d11167 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:39.766572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37ebd0-d666-46e2-9270-4d26c1d11167 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:39.766879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37ebd0-d666-46e2-9270-4d26c1d11167 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:39.768449 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37ebd0-d666-46e2-9270-4d26c1d11167 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:39.782517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 192/391] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:39.784612 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ba801-fe2b-49c4-a234-4cecfe96d142 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:39.785524 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ba801-fe2b-49c4-a234-4cecfe96d142 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:39.786012 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ba801-fe2b-49c4-a234-4cecfe96d142 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:39.788818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37ebd0-d666-46e2-9270-4d26c1d11167 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:39.789738 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 200/392] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:39.802050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89791694-a1e3-45ac-b615-0b580d3df9d8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:39.802692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89791694-a1e3-45ac-b615-0b580d3df9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:39.803229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89791694-a1e3-45ac-b615-0b580d3df9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:39.837612 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-89791694-a1e3-45ac-b615-0b580d3df9d8 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:39.838836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-89791694-a1e3-45ac-b615-0b580d3df9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:39.916293 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 201/393] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:39.951124 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89791694-a1e3-45ac-b615-0b580d3df9d8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:39.971506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37ebd0-d666-46e2-9270-4d26c1d11167 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:40.016827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6428e-5b90-45fa-8eb3-9c036d5193a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:40.017952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6428e-5b90-45fa-8eb3-9c036d5193a2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:40.018378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6428e-5b90-45fa-8eb3-9c036d5193a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:40.018784 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6428e-5b90-45fa-8eb3-9c036d5193a2 None admin] PATH_INFO: `/v3/groups/cd82bf61901242f29b02839bb7fb7f51` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:40.020805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6428e-5b90-45fa-8eb3-9c036d5193a2 None admin] RBAC: Authorizing `identity:get_group(group_id=cd82bf61901242f29b02839bb7fb7f51)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:40.022936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6428e-5b90-45fa-8eb3-9c036d5193a2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:40.024813 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 193/394] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 18 10:15:39 2022] GET /identity/v3/groups/cd82bf61901242f29b02839bb7fb7f51 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:40.030844 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6428e-5b90-45fa-8eb3-9c036d5193a2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:40.067189 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:40.075685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89791694-a1e3-45ac-b615-0b580d3df9d8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:40.078035 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:40.113583 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c98ea8-697f-417e-b35b-532cc73902de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:40.114401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c98ea8-697f-417e-b35b-532cc73902de None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:40.114733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c98ea8-697f-417e-b35b-532cc73902de None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:40.115016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c98ea8-697f-417e-b35b-532cc73902de None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:40.127490 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c98ea8-697f-417e-b35b-532cc73902de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:40.136865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-35c98ea8-697f-417e-b35b-532cc73902de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:40.136865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:40.136865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:40.136865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:40.136865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:40.136865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:40.136865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:40.136865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:40.136865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:40.136865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:40.136865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:40.136865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:40.136865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:40.136865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:40.136865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:40.136865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:40.137816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:40.137816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:40.137816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:40.137816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:40.137816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:40.137816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:40.137816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:40.137816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:40.137816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:40.137816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:40.137816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:40.137816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:40.137816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:40.137816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:40.137816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:40.137816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:40.138610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:40.138610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:40.138610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:40.138610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:40.138610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:40.138610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:40.138610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:40.138610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:40.138610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:40.138610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:40.138610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:40.138610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:40.138610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 202/395] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:15:40.155673 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c98ea8-697f-417e-b35b-532cc73902de None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:40.263648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6428e-5b90-45fa-8eb3-9c036d5193a2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:40.268950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c98ea8-697f-417e-b35b-532cc73902de None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:40.327155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227eee9a-c749-4864-a9d2-e99d0256d831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:40.328020 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227eee9a-c749-4864-a9d2-e99d0256d831 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:40.328305 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227eee9a-c749-4864-a9d2-e99d0256d831 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:40.328612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227eee9a-c749-4864-a9d2-e99d0256d831 None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:40.331052 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227eee9a-c749-4864-a9d2-e99d0256d831 None admin] RBAC: Authorizing `identity:get_project(project_id=09d730b36d8f4ab482dc9ed98757bb0b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:40.332518 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfb9e3-addb-4875-b349-35382e8fd6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:40.333181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfb9e3-addb-4875-b349-35382e8fd6de None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:40.333459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfb9e3-addb-4875-b349-35382e8fd6de None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:40.333729 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfb9e3-addb-4875-b349-35382e8fd6de None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:40.334478 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfb9e3-addb-4875-b349-35382e8fd6de None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:40.334782 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfb9e3-addb-4875-b349-35382e8fd6de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:40.336219 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfb9e3-addb-4875-b349-35382e8fd6de None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:40.338422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227eee9a-c749-4864-a9d2-e99d0256d831 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:40.340176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 194/396] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:40 2022] GET /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:40.346035 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227eee9a-c749-4864-a9d2-e99d0256d831 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:40.357763 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 203/397] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:40.358641 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfb9e3-addb-4875-b349-35382e8fd6de None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:40.391773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:40.509845 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfb9e3-addb-4875-b349-35382e8fd6de None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:40.510516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:40.555053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16d783-e32a-49fc-82e1-b173212689d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:40.555842 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16d783-e32a-49fc-82e1-b173212689d9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:40.556135 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16d783-e32a-49fc-82e1-b173212689d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:40.556412 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16d783-e32a-49fc-82e1-b173212689d9 None admin] PATH_INFO: `/v3/groups/cd82bf61901242f29b02839bb7fb7f51` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:40.559209 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227eee9a-c749-4864-a9d2-e99d0256d831 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:40.561368 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e16d783-e32a-49fc-82e1-b173212689d9 None admin] RBAC: Authorizing `identity:get_group(group_id=cd82bf61901242f29b02839bb7fb7f51)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:40.563205 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e16d783-e32a-49fc-82e1-b173212689d9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:40.565174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 204/398] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 18 10:15:40 2022] GET /identity/v3/groups/cd82bf61901242f29b02839bb7fb7f51 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:40.571138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16d783-e32a-49fc-82e1-b173212689d9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:40.597424 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e7bd9-548a-48ac-8d5c-3918c324d26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:40.598784 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e7bd9-548a-48ac-8d5c-3918c324d26c None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:40.599071 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e7bd9-548a-48ac-8d5c-3918c324d26c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:40.599354 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e7bd9-548a-48ac-8d5c-3918c324d26c None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b/groups/cd82bf61901242f29b02839bb7fb7f51/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:40.602182 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e7bd9-548a-48ac-8d5c-3918c324d26c None admin] RBAC: Authorizing `identity:create_grant(project_id=09d730b36d8f4ab482dc9ed98757bb0b, group_id=cd82bf61901242f29b02839bb7fb7f51, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:40.603921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e7bd9-548a-48ac-8d5c-3918c324d26c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:40.643608 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 195/399] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Jul 18 10:15:40 2022] PUT /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b/groups/cd82bf61901242f29b02839bb7fb7f51/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:15:40.689983 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:40.709420 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16d783-e32a-49fc-82e1-b173212689d9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:40.750172 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0cb28-b86b-4ee8-90fa-4c8048c67c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:40.751519 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0cb28-b86b-4ee8-90fa-4c8048c67c34 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:40.752017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0cb28-b86b-4ee8-90fa-4c8048c67c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:40.752541 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0cb28-b86b-4ee8-90fa-4c8048c67c34 None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:40.754041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c0cb28-b86b-4ee8-90fa-4c8048c67c34 None admin] RBAC: Authorizing `identity:get_project(project_id=57b2ea019ba3413bafb7714839f85a22)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:40.755790 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c0cb28-b86b-4ee8-90fa-4c8048c67c34 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:40.758404 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 205/400] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:40 2022] GET /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:40.763960 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e7bd9-548a-48ac-8d5c-3918c324d26c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:40.838588 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e7cc7-69e6-4152-a9c5-0fc357fc9eba None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:40.839081 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e7cc7-69e6-4152-a9c5-0fc357fc9eba None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:40.839081 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e7cc7-69e6-4152-a9c5-0fc357fc9eba None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:40.840700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 206/401] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:40.845641 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b15316-36d6-4b96-abbe-e1110700b5ba None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:40.845935 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b15316-36d6-4b96-abbe-e1110700b5ba None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:40.846218 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b15316-36d6-4b96-abbe-e1110700b5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:40.864290 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-97b15316-36d6-4b96-abbe-e1110700b5ba None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:40.866871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-97b15316-36d6-4b96-abbe-e1110700b5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:40.918883 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e7bd9-548a-48ac-8d5c-3918c324d26c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:40.969208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804c27a-90c4-4480-adae-ca12761104e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:40.970168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804c27a-90c4-4480-adae-ca12761104e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:40.970449 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804c27a-90c4-4480-adae-ca12761104e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:40.973473 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804c27a-90c4-4480-adae-ca12761104e1 None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22/groups/cd82bf61901242f29b02839bb7fb7f51/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:40.976172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804c27a-90c4-4480-adae-ca12761104e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=57b2ea019ba3413bafb7714839f85a22, group_id=cd82bf61901242f29b02839bb7fb7f51, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:40.977692 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804c27a-90c4-4480-adae-ca12761104e1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:40.980582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 207/402] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:41.002957 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b15316-36d6-4b96-abbe-e1110700b5ba None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:41.006049 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 196/403] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Jul 18 10:15:40 2022] PUT /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22/groups/cd82bf61901242f29b02839bb7fb7f51/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:15:41.164876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:41.193832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b15316-36d6-4b96-abbe-e1110700b5ba None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:41.234222 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cf6e2-0c66-468a-8062-c0a820a80b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:41.235395 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cf6e2-0c66-468a-8062-c0a820a80b7c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:41.235895 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cf6e2-0c66-468a-8062-c0a820a80b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:41.236382 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cf6e2-0c66-468a-8062-c0a820a80b7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:41.248062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cf6e2-0c66-468a-8062-c0a820a80b7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:41.256273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a21cf6e2-0c66-468a-8062-c0a820a80b7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:15:41.256273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:41.256273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:41.256273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:41.256273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:41.256273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:41.256273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:41.256273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:41.256273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:41.256273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:41.256273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:41.256273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:41.256273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:41.256273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:41.256273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:41.256273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:41.257820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:41.257820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:41.257820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:41.257820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:41.257820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:41.257820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:41.257820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:41.257820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:41.257820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:41.257820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:41.257820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:41.257820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:41.257820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:41.257820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:41.257820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:41.257820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:41.258651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:41.258651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:41.258651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:41.258651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:41.258651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:41.258651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:41.258651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:41.258651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:41.258651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:41.258651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:41.258651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:15:41.258651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:41.261375 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 208/404] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:15:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:15:41.268076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804c27a-90c4-4480-adae-ca12761104e1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:41.274289 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:41.301808 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c76f1b-1578-4f1c-a176-ce8ed619fb70 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:41.302466 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c76f1b-1578-4f1c-a176-ce8ed619fb70 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:41.302967 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c76f1b-1578-4f1c-a176-ce8ed619fb70 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:41.305502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 209/405] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:41.310901 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae38ea-e499-4156-8798-110cc5755ac2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:41.311448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae38ea-e499-4156-8798-110cc5755ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:41.311964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae38ea-e499-4156-8798-110cc5755ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:41.333117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-47ae38ea-e499-4156-8798-110cc5755ac2 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:41.334307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-47ae38ea-e499-4156-8798-110cc5755ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:41.405299 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804c27a-90c4-4480-adae-ca12761104e1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:41.442422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9705660-dbd2-4764-b17d-5b6ab7724f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:41.443555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9705660-dbd2-4764-b17d-5b6ab7724f6c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:41.444057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9705660-dbd2-4764-b17d-5b6ab7724f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:41.445229 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9705660-dbd2-4764-b17d-5b6ab7724f6c None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:41.446195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9705660-dbd2-4764-b17d-5b6ab7724f6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:41.446719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9705660-dbd2-4764-b17d-5b6ab7724f6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:41.448065 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 210/406] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:41.448568 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9705660-dbd2-4764-b17d-5b6ab7724f6c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:41.465752 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 197/407] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:41.471366 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae38ea-e499-4156-8798-110cc5755ac2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:41.482998 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9705660-dbd2-4764-b17d-5b6ab7724f6c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:41.551928 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:41.571440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:41.607131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae38ea-e499-4156-8798-110cc5755ac2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:41.633313 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9705660-dbd2-4764-b17d-5b6ab7724f6c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:41.668573 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9b66a-77dc-47f0-8762-89959b395a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:41.669857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9b66a-77dc-47f0-8762-89959b395a66 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:41.673391 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9b66a-77dc-47f0-8762-89959b395a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:41.673884 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9b66a-77dc-47f0-8762-89959b395a66 None admin] PATH_INFO: `/v3/groups/d4f3ac68087d4d788393b4b51a0d096c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:41.675338 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9b66a-77dc-47f0-8762-89959b395a66 None admin] RBAC: Authorizing `identity:get_group(group_id=d4f3ac68087d4d788393b4b51a0d096c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:41.677031 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9b66a-77dc-47f0-8762-89959b395a66 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:41.681638 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 211/408] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 18 10:15:41 2022] GET /identity/v3/groups/d4f3ac68087d4d788393b4b51a0d096c => generated 229 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:41.688111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9b66a-77dc-47f0-8762-89959b395a66 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:41.708024 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc62fa8-482e-4366-9f46-3186b4c7654a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:41.709252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc62fa8-482e-4366-9f46-3186b4c7654a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:41.709802 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc62fa8-482e-4366-9f46-3186b4c7654a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:41.710314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc62fa8-482e-4366-9f46-3186b4c7654a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:41.730494 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc62fa8-482e-4366-9f46-3186b4c7654a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:41.739231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-6cc62fa8-482e-4366-9f46-3186b4c7654a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:41.739231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:41.739231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:41.739231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:41.739231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:41.739231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:41.739231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:41.739231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:41.739231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:41.739231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:41.739231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:41.739231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:41.739231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:41.739231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:41.739231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:41.739231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:41.740164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:41.740164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:41.740164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:41.740164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:41.740164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:41.740164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:41.740164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:41.740164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:41.740164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:41.740164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:41.740164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:41.740164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:41.740164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:41.740164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:41.740164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:41.740164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:41.741033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:41.741033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:41.741033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:41.741033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:41.741033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:41.741033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:41.741033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:41.741033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:41.741033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:41.741033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:41.741033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:41.741033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:41.744278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 198/409] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:15:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:41.752155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc62fa8-482e-4366-9f46-3186b4c7654a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:41.884549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc62fa8-482e-4366-9f46-3186b4c7654a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:41.911898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:41.924819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30609f6-bf79-4d42-a9f4-851da3c6f1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:41.925924 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30609f6-bf79-4d42-a9f4-851da3c6f1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:41.926426 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30609f6-bf79-4d42-a9f4-851da3c6f1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:41.926907 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30609f6-bf79-4d42-a9f4-851da3c6f1c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:41.927867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30609f6-bf79-4d42-a9f4-851da3c6f1c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:41.928389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30609f6-bf79-4d42-a9f4-851da3c6f1c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:41.930423 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30609f6-bf79-4d42-a9f4-851da3c6f1c6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:41.939134 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 199/410] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:15:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:41.946282 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9b66a-77dc-47f0-8762-89959b395a66 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:41.947745 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30609f6-bf79-4d42-a9f4-851da3c6f1c6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:42.050431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1a784-f35f-4730-8fba-e5747c6846e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:42.051313 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1a784-f35f-4730-8fba-e5747c6846e3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:42.051595 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1a784-f35f-4730-8fba-e5747c6846e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:42.051869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1a784-f35f-4730-8fba-e5747c6846e3 None admin] PATH_INFO: `/v3/projects/1282af50d400435ba3f0e00de6e5fb67` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:42.055318 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee1a784-f35f-4730-8fba-e5747c6846e3 None admin] RBAC: Authorizing `identity:get_project(project_id=1282af50d400435ba3f0e00de6e5fb67)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:42.056797 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee1a784-f35f-4730-8fba-e5747c6846e3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:42.058822 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 212/411] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:41 2022] GET /identity/v3/projects/1282af50d400435ba3f0e00de6e5fb67 => generated 344 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:42.069536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1a784-f35f-4730-8fba-e5747c6846e3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:42.077907 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30609f6-bf79-4d42-a9f4-851da3c6f1c6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:42.092846 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:42.113472 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a251af-1f70-4eba-9e49-48c8af2c0d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:42.114693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a251af-1f70-4eba-9e49-48c8af2c0d69 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:42.115185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a251af-1f70-4eba-9e49-48c8af2c0d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:42.115655 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a251af-1f70-4eba-9e49-48c8af2c0d69 None admin] PATH_INFO: `/v3/groups/cd82bf61901242f29b02839bb7fb7f51` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:42.117614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a251af-1f70-4eba-9e49-48c8af2c0d69 None admin] RBAC: Authorizing `identity:get_group(group_id=cd82bf61901242f29b02839bb7fb7f51)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:42.119332 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a251af-1f70-4eba-9e49-48c8af2c0d69 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:42.121553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 200/412] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 18 10:15:41 2022] GET /identity/v3/groups/cd82bf61901242f29b02839bb7fb7f51 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:42.129824 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a251af-1f70-4eba-9e49-48c8af2c0d69 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:42.201502 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:42.212669 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1a784-f35f-4730-8fba-e5747c6846e3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:42.255378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a251af-1f70-4eba-9e49-48c8af2c0d69 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:42.257993 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833c016-dbf0-431a-b69c-cd006a914c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:42.259137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833c016-dbf0-431a-b69c-cd006a914c5c None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:42.259539 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833c016-dbf0-431a-b69c-cd006a914c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:42.259915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833c016-dbf0-431a-b69c-cd006a914c5c None admin] PATH_INFO: `/v3/projects/1282af50d400435ba3f0e00de6e5fb67/groups/d4f3ac68087d4d788393b4b51a0d096c/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:42.262956 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833c016-dbf0-431a-b69c-cd006a914c5c None admin] RBAC: Authorizing `identity:create_grant(project_id=1282af50d400435ba3f0e00de6e5fb67, group_id=d4f3ac68087d4d788393b4b51a0d096c, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:42.264807 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833c016-dbf0-431a-b69c-cd006a914c5c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:42.283458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 213/413] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Jul 18 10:15:42 2022] PUT /identity/v3/projects/1282af50d400435ba3f0e00de6e5fb67/groups/d4f3ac68087d4d788393b4b51a0d096c/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:15:42.286318 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786bbae-da06-48fd-9dac-70f6c055d842 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:42.287331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786bbae-da06-48fd-9dac-70f6c055d842 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:42.287631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786bbae-da06-48fd-9dac-70f6c055d842 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:42.290106 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 214/414] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:42.295950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd89061-4a37-475f-81f2-f2316da815ff None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:42.296263 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd89061-4a37-475f-81f2-f2316da815ff None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:42.296535 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd89061-4a37-475f-81f2-f2316da815ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:42.299620 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6a7ec-6459-48f0-ad08-f1b27bdc419f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:42.300540 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6a7ec-6459-48f0-ad08-f1b27bdc419f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:42.300890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6a7ec-6459-48f0-ad08-f1b27bdc419f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:42.301227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6a7ec-6459-48f0-ad08-f1b27bdc419f None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:42.302480 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6a7ec-6459-48f0-ad08-f1b27bdc419f None admin] RBAC: Authorizing `identity:get_project(project_id=57b2ea019ba3413bafb7714839f85a22)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:42.304115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6a7ec-6459-48f0-ad08-f1b27bdc419f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:42.305985 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 201/415] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:42 2022] GET /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:42.314006 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6a7ec-6459-48f0-ad08-f1b27bdc419f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:42.319845 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8fd89061-4a37-475f-81f2-f2316da815ff None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:42.321146 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8fd89061-4a37-475f-81f2-f2316da815ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:42.434451 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6a7ec-6459-48f0-ad08-f1b27bdc419f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:42.448358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:42.454967 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 215/416] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:42.479970 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd89061-4a37-475f-81f2-f2316da815ff None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:42.484860 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:42.500424 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bdb6e-55f3-4eda-a053-6f48c636170f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:42.501782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bdb6e-55f3-4eda-a053-6f48c636170f None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:42.502076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bdb6e-55f3-4eda-a053-6f48c636170f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:42.502355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bdb6e-55f3-4eda-a053-6f48c636170f None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22/groups/cd82bf61901242f29b02839bb7fb7f51/roles/8c039234db9640d0bdc0c2a629ff9e52` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:42.509156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bdb6e-55f3-4eda-a053-6f48c636170f None admin] RBAC: Authorizing `identity:create_grant(project_id=57b2ea019ba3413bafb7714839f85a22, group_id=cd82bf61901242f29b02839bb7fb7f51, role_id=8c039234db9640d0bdc0c2a629ff9e52)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:42.510631 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bdb6e-55f3-4eda-a053-6f48c636170f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:42.540891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 202/417] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Jul 18 10:15:42 2022] PUT /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22/groups/cd82bf61901242f29b02839bb7fb7f51/roles/8c039234db9640d0bdc0c2a629ff9e52 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:15:42.608885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd89061-4a37-475f-81f2-f2316da815ff None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:42.648270 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9fd63-a93c-49ce-9746-102a692a1cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:42.649124 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9fd63-a93c-49ce-9746-102a692a1cbf None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:42.649404 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9fd63-a93c-49ce-9746-102a692a1cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:42.649674 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9fd63-a93c-49ce-9746-102a692a1cbf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:42.661691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9fd63-a93c-49ce-9746-102a692a1cbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:42.670070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-81c9fd63-a93c-49ce-9746-102a692a1cbf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:42.670070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:42.670070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:42.670070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:42.670070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:42.670070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:42.670070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:42.670070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:42.670070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:42.670070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:42.670070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:42.670070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:42.670070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:42.670070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:42.670070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:42.670070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:42.671016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:42.671016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:42.671016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:42.671016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:42.671016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:42.671016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:42.671016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:42.671016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:42.671016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:42.671016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:42.671016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:42.671016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:42.671016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:42.671016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:42.671016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:42.671016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:42.672691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:42.672691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:42.672691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:42.672691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:42.672691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:42.672691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:42.672691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:42.672691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:42.672691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:42.672691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:42.672691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:42.672691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:42.672691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 216/418] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:15:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:42.677629 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bdb6e-55f3-4eda-a053-6f48c636170f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:42.801410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bdb6e-55f3-4eda-a053-6f48c636170f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:42.837069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def6068-37e1-4049-9f6d-feb69858de14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:42.837744 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def6068-37e1-4049-9f6d-feb69858de14 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:42.838027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def6068-37e1-4049-9f6d-feb69858de14 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:42.838303 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def6068-37e1-4049-9f6d-feb69858de14 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:42.839059 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def6068-37e1-4049-9f6d-feb69858de14 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:42.839366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def6068-37e1-4049-9f6d-feb69858de14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:42.840755 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def6068-37e1-4049-9f6d-feb69858de14 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:42.849782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 203/419] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:15:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:42.855540 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9fd63-a93c-49ce-9746-102a692a1cbf None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:42.926926 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:42.980820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9fd63-a93c-49ce-9746-102a692a1cbf None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:43.024852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3f7d2-8b23-4b0b-b17a-754c16f19d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:43.025520 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3f7d2-8b23-4b0b-b17a-754c16f19d31 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:43.025806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3f7d2-8b23-4b0b-b17a-754c16f19d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:43.026081 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3f7d2-8b23-4b0b-b17a-754c16f19d31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:43.114949 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3f7d2-8b23-4b0b-b17a-754c16f19d31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:43.116477 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3f7d2-8b23-4b0b-b17a-754c16f19d31 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:43.189657 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:43.240177 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 217/420] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:43.245772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def6068-37e1-4049-9f6d-feb69858de14 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:43.287354 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639eb52-e63f-4304-97ee-750bd270541a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:43.287663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639eb52-e63f-4304-97ee-750bd270541a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:43.287919 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639eb52-e63f-4304-97ee-750bd270541a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:43.290003 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 218/421] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:43.295189 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775efdff-7d91-447c-ac1b-2ad0b26d9c25 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:43.295701 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775efdff-7d91-447c-ac1b-2ad0b26d9c25 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:43.296186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775efdff-7d91-447c-ac1b-2ad0b26d9c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:43.312875 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-775efdff-7d91-447c-ac1b-2ad0b26d9c25 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:43.314276 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:43.314645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-775efdff-7d91-447c-ac1b-2ad0b26d9c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:43.393706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def6068-37e1-4049-9f6d-feb69858de14 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:43.427717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 219/422] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:43.428433 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3322ebe-5370-47de-9acc-476df2b77386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:43.429364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3322ebe-5370-47de-9acc-476df2b77386 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:43.429648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3322ebe-5370-47de-9acc-476df2b77386 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:43.429923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3322ebe-5370-47de-9acc-476df2b77386 None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:43.431385 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3322ebe-5370-47de-9acc-476df2b77386 None admin] RBAC: Authorizing `identity:get_project(project_id=09d730b36d8f4ab482dc9ed98757bb0b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:43.434753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3322ebe-5370-47de-9acc-476df2b77386 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:43.434753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 204/423] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:43 2022] GET /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:43.439845 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775efdff-7d91-447c-ac1b-2ad0b26d9c25 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:43.452730 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3322ebe-5370-47de-9acc-476df2b77386 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:43.577833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3322ebe-5370-47de-9acc-476df2b77386 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:43.592948 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:43.596944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:43.609138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775efdff-7d91-447c-ac1b-2ad0b26d9c25 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:43.613565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46aa6ff-7f57-4c86-8d62-c2cd18a62a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:43.614470 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46aa6ff-7f57-4c86-8d62-c2cd18a62a5b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:43.614853 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46aa6ff-7f57-4c86-8d62-c2cd18a62a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:43.615225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46aa6ff-7f57-4c86-8d62-c2cd18a62a5b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:43.626441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46aa6ff-7f57-4c86-8d62-c2cd18a62a5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:43.638898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f46aa6ff-7f57-4c86-8d62-c2cd18a62a5b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:43.638898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:43.638898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:43.638898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:43.638898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:43.638898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:43.638898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:43.638898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:43.638898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:43.638898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:43.638898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:43.638898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:43.638898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:43.638898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:43.638898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:43.638898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:43.639816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:43.639816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:43.639816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:43.639816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:43.639816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:43.639816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:43.639816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:43.639816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:43.639816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:43.639816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:43.639816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:43.639816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:43.639816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:43.639816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:43.639816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:43.639816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:43.640651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:43.640651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:43.640651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:43.640651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:43.640651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:43.640651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:43.640651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:43.640651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:43.640651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:43.640651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:43.640651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:43.640651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:43.641272 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 205/424] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:15:43.642230 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cb828-eca0-4d42-bb25-3f88005553ab None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:43.642521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cb828-eca0-4d42-bb25-3f88005553ab None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:43.642801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cb828-eca0-4d42-bb25-3f88005553ab None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:43.648080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 206/425] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:43.648080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cb828-eca0-4d42-bb25-3f88005553ab None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:43.660367 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fd6cd-5a20-41c6-9dfa-b62c3bacb90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:43.661947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fd6cd-5a20-41c6-9dfa-b62c3bacb90b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:43.662445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fd6cd-5a20-41c6-9dfa-b62c3bacb90b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:43.662919 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fd6cd-5a20-41c6-9dfa-b62c3bacb90b None admin] PATH_INFO: `/v3/groups/cd82bf61901242f29b02839bb7fb7f51` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:43.664389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6fd6cd-5a20-41c6-9dfa-b62c3bacb90b None admin] RBAC: Authorizing `identity:get_group(group_id=cd82bf61901242f29b02839bb7fb7f51)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:43.666609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6fd6cd-5a20-41c6-9dfa-b62c3bacb90b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:43.669824 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 220/426] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 18 10:15:43 2022] GET /identity/v3/groups/cd82bf61901242f29b02839bb7fb7f51 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:43.671622 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5ff76-5deb-4355-ad3a-043dc62479a7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:43.672234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5ff76-5deb-4355-ad3a-043dc62479a7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:43.673051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5ff76-5deb-4355-ad3a-043dc62479a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:43.695703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1cc5ff76-5deb-4355-ad3a-043dc62479a7 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:43.696940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1cc5ff76-5deb-4355-ad3a-043dc62479a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:43.765214 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cb828-eca0-4d42-bb25-3f88005553ab None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:43.768806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 221/427] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:43.769682 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc5ff76-5deb-4355-ad3a-043dc62479a7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:43.807804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcce391-34ef-4b7f-b46f-42bae105fbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:43.808540 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcce391-34ef-4b7f-b46f-42bae105fbdb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:43.808813 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcce391-34ef-4b7f-b46f-42bae105fbdb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:43.809091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcce391-34ef-4b7f-b46f-42bae105fbdb None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:43.809840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcce391-34ef-4b7f-b46f-42bae105fbdb None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:43.810147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcce391-34ef-4b7f-b46f-42bae105fbdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:43.811647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcce391-34ef-4b7f-b46f-42bae105fbdb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:43.822113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 207/428] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:43.827704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcce391-34ef-4b7f-b46f-42bae105fbdb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:43.865926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:43.875246 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:43.894911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc5ff76-5deb-4355-ad3a-043dc62479a7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:43.966842 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcce391-34ef-4b7f-b46f-42bae105fbdb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:43.980527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3b362-648b-4a95-8cf3-362401e4d876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:43.980527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3b362-648b-4a95-8cf3-362401e4d876 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:43.980527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3b362-648b-4a95-8cf3-362401e4d876 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:43.980527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3b362-648b-4a95-8cf3-362401e4d876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:43.980939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3b362-648b-4a95-8cf3-362401e4d876 None admin] RBAC: Adding query filter params (group.id=cd82bf61901242f29b02839bb7fb7f51, role.id=8c039234db9640d0bdc0c2a629ff9e52, scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:43.980939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3b362-648b-4a95-8cf3-362401e4d876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:43.984532 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3b362-648b-4a95-8cf3-362401e4d876 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:43.989423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 222/429] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 18 10:15:43 2022] GET /identity/v3/role_assignments?group.id=cd82bf61901242f29b02839bb7fb7f51&scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b&role.id=8c039234db9640d0bdc0c2a629ff9e52 => generated 611 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:43.992545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3b362-648b-4a95-8cf3-362401e4d876 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:44.067507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568e5fb-6c02-43b1-90c9-8f0aa7a9c580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:44.067507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568e5fb-6c02-43b1-90c9-8f0aa7a9c580 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:44.067507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568e5fb-6c02-43b1-90c9-8f0aa7a9c580 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:44.067507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568e5fb-6c02-43b1-90c9-8f0aa7a9c580 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:44.088530 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568e5fb-6c02-43b1-90c9-8f0aa7a9c580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:44.100359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a568e5fb-6c02-43b1-90c9-8f0aa7a9c580 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:44.100359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:44.100359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:44.100359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:44.100359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:44.100359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:44.100359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:44.100359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:44.100359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:44.100359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:44.100359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:44.100359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:44.100359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:44.100359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:44.100359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:44.100359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:44.102201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:44.102201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:44.102201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:44.102201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:44.102201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:44.102201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:44.102201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:44.102201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:44.102201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:44.102201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:44.102201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:44.102201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:44.102201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:44.102201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:44.102201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:44.102201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:44.103572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:44.103572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:44.103572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:44.103572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:44.103572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:44.103572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:44.103572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:44.103572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:44.103572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:44.103572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:44.103572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 10:15:44.103572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:44.117673 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 208/430] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:44.118671 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568e5fb-6c02-43b1-90c9-8f0aa7a9c580 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:44.200886 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3b362-648b-4a95-8cf3-362401e4d876 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:44.248745 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6dae4-8983-4856-bc11-7331c7bf15b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:44.249386 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6dae4-8983-4856-bc11-7331c7bf15b5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:44.249699 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6dae4-8983-4856-bc11-7331c7bf15b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:44.250401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6dae4-8983-4856-bc11-7331c7bf15b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:44.263335 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:44.278665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568e5fb-6c02-43b1-90c9-8f0aa7a9c580 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:44.280997 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:44.345826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6dae4-8983-4856-bc11-7331c7bf15b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:44.348918 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6dae4-8983-4856-bc11-7331c7bf15b5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:44.358584 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5065d-9423-4889-8a75-c968159cdb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:44.359564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5065d-9423-4889-8a75-c968159cdb32 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:44.360113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5065d-9423-4889-8a75-c968159cdb32 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:44.363729 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5065d-9423-4889-8a75-c968159cdb32 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:44.365780 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5065d-9423-4889-8a75-c968159cdb32 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:44.367084 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5065d-9423-4889-8a75-c968159cdb32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:44.371660 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5065d-9423-4889-8a75-c968159cdb32 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:44.400298 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 209/431] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:15:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:44.408893 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5065d-9423-4889-8a75-c968159cdb32 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:44.510880 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 223/432] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:44.521376 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6dae4-8983-4856-bc11-7331c7bf15b5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:44.563221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5065d-9423-4889-8a75-c968159cdb32 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:44.608159 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e36943-0d95-489c-bc07-9d5bb1dda604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:44.608849 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e36943-0d95-489c-bc07-9d5bb1dda604 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:44.609137 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e36943-0d95-489c-bc07-9d5bb1dda604 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:44.609426 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e36943-0d95-489c-bc07-9d5bb1dda604 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:44.609905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:44.649573 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:44.658007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6dae4-8983-4856-bc11-7331c7bf15b5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:44.695950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495858a-7cba-4b09-8d1c-c3e51b358a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:44.697338 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495858a-7cba-4b09-8d1c-c3e51b358a6c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:44.697821 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495858a-7cba-4b09-8d1c-c3e51b358a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:44.698312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495858a-7cba-4b09-8d1c-c3e51b358a6c None admin] PATH_INFO: `/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:44.699710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495858a-7cba-4b09-8d1c-c3e51b358a6c None admin] RBAC: Authorizing `identity:get_project(project_id=09d730b36d8f4ab482dc9ed98757bb0b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:44.701382 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495858a-7cba-4b09-8d1c-c3e51b358a6c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:44.703287 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 224/433] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:44 2022] GET /identity/v3/projects/09d730b36d8f4ab482dc9ed98757bb0b => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:44.707815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e36943-0d95-489c-bc07-9d5bb1dda604 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:44.707815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e36943-0d95-489c-bc07-9d5bb1dda604 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:44.709031 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495858a-7cba-4b09-8d1c-c3e51b358a6c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:44.822712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 210/434] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:44.828240 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e36943-0d95-489c-bc07-9d5bb1dda604 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:44.835446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495858a-7cba-4b09-8d1c-c3e51b358a6c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:44.857911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:44.877394 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30afc1-1097-4e35-b195-774182ed3fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:44.878388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30afc1-1097-4e35-b195-774182ed3fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:44.878680 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30afc1-1097-4e35-b195-774182ed3fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:44.878957 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30afc1-1097-4e35-b195-774182ed3fa7 None admin] PATH_INFO: `/v3/groups/cd82bf61901242f29b02839bb7fb7f51` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:44.880144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30afc1-1097-4e35-b195-774182ed3fa7 None admin] RBAC: Authorizing `identity:get_group(group_id=cd82bf61901242f29b02839bb7fb7f51)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:44.881624 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30afc1-1097-4e35-b195-774182ed3fa7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:44.883470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 225/435] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 18 10:15:44 2022] GET /identity/v3/groups/cd82bf61901242f29b02839bb7fb7f51 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:44.884876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcd813-3ea6-4731-807c-e0f4dceea459 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:44.885168 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcd813-3ea6-4731-807c-e0f4dceea459 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:44.885451 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcd813-3ea6-4731-807c-e0f4dceea459 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:44.887108 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 226/436] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:44.890531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcd813-3ea6-4731-807c-e0f4dceea459 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:44.894215 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:44.939856 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e36943-0d95-489c-bc07-9d5bb1dda604 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:44.974642 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af864a2-c20f-4358-91dd-094d950efa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:44.975513 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af864a2-c20f-4358-91dd-094d950efa58 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:44.976349 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af864a2-c20f-4358-91dd-094d950efa58 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:44.977074 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af864a2-c20f-4358-91dd-094d950efa58 None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:44.978290 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af864a2-c20f-4358-91dd-094d950efa58 None admin] RBAC: Authorizing `identity:get_project(project_id=57b2ea019ba3413bafb7714839f85a22)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:44.980704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af864a2-c20f-4358-91dd-094d950efa58 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:44.982687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 211/437] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:44 2022] GET /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:44.987955 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b262489-9bf4-4faa-9b77-e7d4ac72df4e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:44.988246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b262489-9bf4-4faa-9b77-e7d4ac72df4e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:44.988544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b262489-9bf4-4faa-9b77-e7d4ac72df4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:45.004411 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcd813-3ea6-4731-807c-e0f4dceea459 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:45.005156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9b262489-9bf4-4faa-9b77-e7d4ac72df4e None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:45.006335 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9b262489-9bf4-4faa-9b77-e7d4ac72df4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:45.041587 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5603e-96a7-4150-a41a-346d2521e002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:45.042230 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5603e-96a7-4150-a41a-346d2521e002 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:45.042517 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5603e-96a7-4150-a41a-346d2521e002 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:45.042795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5603e-96a7-4150-a41a-346d2521e002 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:45.043629 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5603e-96a7-4150-a41a-346d2521e002 None admin] RBAC: Adding query filter params (group.id=cd82bf61901242f29b02839bb7fb7f51, role.id=2820090979cb487db491f835e03e6471, scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:45.043951 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5603e-96a7-4150-a41a-346d2521e002 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:45.045426 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5603e-96a7-4150-a41a-346d2521e002 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:45.051483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 227/438] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 18 10:15:44 2022] GET /identity/v3/role_assignments?group.id=cd82bf61901242f29b02839bb7fb7f51&scope.project.id=09d730b36d8f4ab482dc9ed98757bb0b&role.id=2820090979cb487db491f835e03e6471 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:45.052283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5603e-96a7-4150-a41a-346d2521e002 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:45.096575 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 212/439] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:45.097692 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a6799-e115-43b4-b54e-bf33efd32dd0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:45.097992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a6799-e115-43b4-b54e-bf33efd32dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:45.098272 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a6799-e115-43b4-b54e-bf33efd32dd0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:45.099671 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 213/440] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:45.105548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439d7bc-ad68-45eb-b418-cd56f699a1cb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:45.105840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439d7bc-ad68-45eb-b418-cd56f699a1cb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:45.106127 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439d7bc-ad68-45eb-b418-cd56f699a1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:45.118064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:45.129729 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2439d7bc-ad68-45eb-b418-cd56f699a1cb None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:45.131073 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2439d7bc-ad68-45eb-b418-cd56f699a1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:45.183314 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5603e-96a7-4150-a41a-346d2521e002 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:45.189641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:45.212669 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 214/441] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:45.212669 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439d7bc-ad68-45eb-b418-cd56f699a1cb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:45.224412 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010844d8-02ed-41be-bd71-45ea88993eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:45.225505 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010844d8-02ed-41be-bd71-45ea88993eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:45.226077 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010844d8-02ed-41be-bd71-45ea88993eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:45.226507 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010844d8-02ed-41be-bd71-45ea88993eb5 None admin] PATH_INFO: `/v3/groups/cd82bf61901242f29b02839bb7fb7f51` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:45.227875 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010844d8-02ed-41be-bd71-45ea88993eb5 None admin] RBAC: Authorizing `identity:get_group(group_id=cd82bf61901242f29b02839bb7fb7f51)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:45.230590 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010844d8-02ed-41be-bd71-45ea88993eb5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:45.232718 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 228/442] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 18 10:15:45 2022] GET /identity/v3/groups/cd82bf61901242f29b02839bb7fb7f51 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:45.234595 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010844d8-02ed-41be-bd71-45ea88993eb5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:45.336991 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439d7bc-ad68-45eb-b418-cd56f699a1cb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:45.370537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010844d8-02ed-41be-bd71-45ea88993eb5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:45.374847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f007a5f-7eb2-449a-9f64-7587a047e32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:45.375654 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f007a5f-7eb2-449a-9f64-7587a047e32b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:45.375942 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f007a5f-7eb2-449a-9f64-7587a047e32b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:45.376223 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f007a5f-7eb2-449a-9f64-7587a047e32b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:45.388231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f007a5f-7eb2-449a-9f64-7587a047e32b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:45.390223 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:45.396619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9f007a5f-7eb2-449a-9f64-7587a047e32b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:15:45.396619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:45.396619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:45.396619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:45.396619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:45.396619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:45.396619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:45.396619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:45.396619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:45.396619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:45.396619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:45.396619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:45.396619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:45.396619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:45.396619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:45.396619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:45.397598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:45.397598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:45.397598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:45.397598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:45.397598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:45.397598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:45.397598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:45.397598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:45.397598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:45.397598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:45.397598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:45.397598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:45.397598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:45.397598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:45.397598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:45.397598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:45.398585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:45.398585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:45.398585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:45.398585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:45.398585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:45.398585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:45.398585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:45.398585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:45.398585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:45.398585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:45.398585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:15:45.398585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:45.398585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 215/443] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:15:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:45.399404 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f007a5f-7eb2-449a-9f64-7587a047e32b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:45.413980 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a3a71-2d0a-40a3-9f40-cd4f43ee85ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:45.414924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a3a71-2d0a-40a3-9f40-cd4f43ee85ae None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:45.415212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a3a71-2d0a-40a3-9f40-cd4f43ee85ae None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:45.415678 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a3a71-2d0a-40a3-9f40-cd4f43ee85ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:45.426221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a3a71-2d0a-40a3-9f40-cd4f43ee85ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:45.434565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ee1a3a71-2d0a-40a3-9f40-cd4f43ee85ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:45.434565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:45.434565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:45.434565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:45.434565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:45.434565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:45.434565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:45.434565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:45.434565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:45.434565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:45.434565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:45.434565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:45.434565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:45.434565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:45.434565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:45.434565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:45.435427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:45.435427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:45.435427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:45.435427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:45.435427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:45.435427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:45.435427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:45.435427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:45.435427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:45.435427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:45.435427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:45.435427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:45.435427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:45.435427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:45.435427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:45.435427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:45.436453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:45.436453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:45.436453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:45.436453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:45.436453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:45.436453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:45.436453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:45.436453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:45.436453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:45.436453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:45.436453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:45.436453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:45.436453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 229/444] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:15:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:15:45.437504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a3a71-2d0a-40a3-9f40-cd4f43ee85ae None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:45.457519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:45.525499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f007a5f-7eb2-449a-9f64-7587a047e32b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:45.565519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2521ce-39fd-4592-ad13-f6edede21941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:45.566183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2521ce-39fd-4592-ad13-f6edede21941 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:45.566472 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2521ce-39fd-4592-ad13-f6edede21941 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:45.566754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2521ce-39fd-4592-ad13-f6edede21941 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:45.567605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2521ce-39fd-4592-ad13-f6edede21941 None admin] RBAC: Adding query filter params (group.id=cd82bf61901242f29b02839bb7fb7f51, role.id=2820090979cb487db491f835e03e6471, scope.project.id=57b2ea019ba3413bafb7714839f85a22) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:45.567827 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a3a71-2d0a-40a3-9f40-cd4f43ee85ae None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:45.567923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2521ce-39fd-4592-ad13-f6edede21941 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:45.569995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2521ce-39fd-4592-ad13-f6edede21941 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:45.577648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 216/445] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 18 10:15:45 2022] GET /identity/v3/role_assignments?group.id=cd82bf61901242f29b02839bb7fb7f51&scope.project.id=57b2ea019ba3413bafb7714839f85a22&role.id=2820090979cb487db491f835e03e6471 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:45.578484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2521ce-39fd-4592-ad13-f6edede21941 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:45.625398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d28f4-4f5c-4037-8a15-eca95ce44ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:45.626007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d28f4-4f5c-4037-8a15-eca95ce44ffb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:45.626295 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d28f4-4f5c-4037-8a15-eca95ce44ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:45.626574 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d28f4-4f5c-4037-8a15-eca95ce44ffb None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:45.627340 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d28f4-4f5c-4037-8a15-eca95ce44ffb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:45.627695 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d28f4-4f5c-4037-8a15-eca95ce44ffb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:45.629182 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d28f4-4f5c-4037-8a15-eca95ce44ffb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:45.640957 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 230/446] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:45.647113 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d28f4-4f5c-4037-8a15-eca95ce44ffb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:45.694261 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:45.704330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2521ce-39fd-4592-ad13-f6edede21941 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:45.715279 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:45.749829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f7b4d-2dea-4d78-a1bd-c5cdc2799353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:45.750724 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f7b4d-2dea-4d78-a1bd-c5cdc2799353 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:45.750862 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f7b4d-2dea-4d78-a1bd-c5cdc2799353 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:45.751052 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f7b4d-2dea-4d78-a1bd-c5cdc2799353 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:45.751803 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f7b4d-2dea-4d78-a1bd-c5cdc2799353 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:45.752114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f7b4d-2dea-4d78-a1bd-c5cdc2799353 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:45.753528 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f7b4d-2dea-4d78-a1bd-c5cdc2799353 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:45.764462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 217/447] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:15:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:45.765188 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d28f4-4f5c-4037-8a15-eca95ce44ffb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:45.770087 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f7b4d-2dea-4d78-a1bd-c5cdc2799353 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:45.806366 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b70376-dda3-42f4-9d49-64fc0e3c87bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:45.807036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b70376-dda3-42f4-9d49-64fc0e3c87bb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:45.807324 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b70376-dda3-42f4-9d49-64fc0e3c87bb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:45.807594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b70376-dda3-42f4-9d49-64fc0e3c87bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:45.885803 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f7b4d-2dea-4d78-a1bd-c5cdc2799353 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:45.893530 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b70376-dda3-42f4-9d49-64fc0e3c87bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:45.895014 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b70376-dda3-42f4-9d49-64fc0e3c87bb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:45.925361 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74f56c-75a8-4023-be33-6fe075ced4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:45.926016 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74f56c-75a8-4023-be33-6fe075ced4ad None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:45.926303 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74f56c-75a8-4023-be33-6fe075ced4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:45.926580 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74f56c-75a8-4023-be33-6fe075ced4ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:45.940793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:45.971882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:46.001996 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74f56c-75a8-4023-be33-6fe075ced4ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:46.003467 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74f56c-75a8-4023-be33-6fe075ced4ad None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:46.005839 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 231/448] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 10:15:46.012043 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b70376-dda3-42f4-9d49-64fc0e3c87bb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:46.099410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 218/449] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 10:15:46.105667 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74f56c-75a8-4023-be33-6fe075ced4ad None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:46.108107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b70376-dda3-42f4-9d49-64fc0e3c87bb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:46.144118 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c2781-12ae-4352-a7bb-f1a34ae6cca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:46.144934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c2781-12ae-4352-a7bb-f1a34ae6cca6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:46.145220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c2781-12ae-4352-a7bb-f1a34ae6cca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:46.145497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c2781-12ae-4352-a7bb-f1a34ae6cca6 None admin] PATH_INFO: `/v3/projects/1282af50d400435ba3f0e00de6e5fb67` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:46.146677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c2781-12ae-4352-a7bb-f1a34ae6cca6 None admin] RBAC: Authorizing `identity:get_project(project_id=1282af50d400435ba3f0e00de6e5fb67)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:46.148214 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c2781-12ae-4352-a7bb-f1a34ae6cca6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:46.150242 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 232/450] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:46 2022] GET /identity/v3/projects/1282af50d400435ba3f0e00de6e5fb67 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:46.155643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c2781-12ae-4352-a7bb-f1a34ae6cca6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:46.166339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:46.176526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:46.209594 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74f56c-75a8-4023-be33-6fe075ced4ad None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:46.243676 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991308f-e65d-4fb3-85cf-70aa1e978ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:46.244560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991308f-e65d-4fb3-85cf-70aa1e978ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:46.244845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991308f-e65d-4fb3-85cf-70aa1e978ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:46.245121 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991308f-e65d-4fb3-85cf-70aa1e978ab6 None admin] PATH_INFO: `/v3/projects/57b2ea019ba3413bafb7714839f85a22` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:46.246302 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991308f-e65d-4fb3-85cf-70aa1e978ab6 None admin] RBAC: Authorizing `identity:get_project(project_id=57b2ea019ba3413bafb7714839f85a22)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:46.247737 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991308f-e65d-4fb3-85cf-70aa1e978ab6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:46.249499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 219/451] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:15:46 2022] GET /identity/v3/projects/57b2ea019ba3413bafb7714839f85a22 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:46.253638 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c2781-12ae-4352-a7bb-f1a34ae6cca6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:46.254950 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991308f-e65d-4fb3-85cf-70aa1e978ab6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:46.291628 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e66801-5d78-4384-9b33-0b05638c929a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:46.292489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e66801-5d78-4384-9b33-0b05638c929a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:46.292801 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e66801-5d78-4384-9b33-0b05638c929a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:46.293072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e66801-5d78-4384-9b33-0b05638c929a None admin] PATH_INFO: `/v3/groups/d4f3ac68087d4d788393b4b51a0d096c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:46.294286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e66801-5d78-4384-9b33-0b05638c929a None admin] RBAC: Authorizing `identity:get_group(group_id=d4f3ac68087d4d788393b4b51a0d096c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:46.295738 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e66801-5d78-4384-9b33-0b05638c929a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:46.297402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 233/452] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 18 10:15:46 2022] GET /identity/v3/groups/d4f3ac68087d4d788393b4b51a0d096c => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:46.303064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e66801-5d78-4384-9b33-0b05638c929a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:46.358931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991308f-e65d-4fb3-85cf-70aa1e978ab6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:46.377772 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:46.393383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2608e-f51f-46b2-bae7-9a86eba6228c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:46.394288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2608e-f51f-46b2-bae7-9a86eba6228c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:46.394516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2608e-f51f-46b2-bae7-9a86eba6228c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:46.394796 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2608e-f51f-46b2-bae7-9a86eba6228c None admin] PATH_INFO: `/v3/groups/cd82bf61901242f29b02839bb7fb7f51` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:46.395972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2608e-f51f-46b2-bae7-9a86eba6228c None admin] RBAC: Authorizing `identity:get_group(group_id=cd82bf61901242f29b02839bb7fb7f51)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:46.397538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2608e-f51f-46b2-bae7-9a86eba6228c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:46.399156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 220/453] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jul 18 10:15:46 2022] GET /identity/v3/groups/cd82bf61901242f29b02839bb7fb7f51 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:46.400405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e66801-5d78-4384-9b33-0b05638c929a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:46.404887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2608e-f51f-46b2-bae7-9a86eba6228c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:46.414202 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:46.434627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bad1d-0e4e-4d98-b2fe-458a2b84272b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:46.435289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bad1d-0e4e-4d98-b2fe-458a2b84272b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:46.435577 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bad1d-0e4e-4d98-b2fe-458a2b84272b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:46.435860 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bad1d-0e4e-4d98-b2fe-458a2b84272b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:46.436721 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bad1d-0e4e-4d98-b2fe-458a2b84272b None admin] RBAC: Adding query filter params (group.id=d4f3ac68087d4d788393b4b51a0d096c, role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.project.id=1282af50d400435ba3f0e00de6e5fb67) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:46.437039 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bad1d-0e4e-4d98-b2fe-458a2b84272b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:46.438470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bad1d-0e4e-4d98-b2fe-458a2b84272b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:46.444275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 234/454] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 18 10:15:46 2022] GET /identity/v3/role_assignments?group.id=d4f3ac68087d4d788393b4b51a0d096c&scope.project.id=1282af50d400435ba3f0e00de6e5fb67&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:46.512622 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2608e-f51f-46b2-bae7-9a86eba6228c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:46.547681 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1a71b-c6d1-4094-b4da-5593ce3dd7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:46.548346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1a71b-c6d1-4094-b4da-5593ce3dd7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:46.548668 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1a71b-c6d1-4094-b4da-5593ce3dd7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:46.548948 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1a71b-c6d1-4094-b4da-5593ce3dd7c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:46.549798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1a71b-c6d1-4094-b4da-5593ce3dd7c4 None admin] RBAC: Adding query filter params (group.id=cd82bf61901242f29b02839bb7fb7f51, role.id=8c039234db9640d0bdc0c2a629ff9e52, scope.project.id=57b2ea019ba3413bafb7714839f85a22) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:46.550115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1a71b-c6d1-4094-b4da-5593ce3dd7c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:46.551582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1a71b-c6d1-4094-b4da-5593ce3dd7c4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:46.558192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 221/455] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jul 18 10:15:46 2022] GET /identity/v3/role_assignments?group.id=cd82bf61901242f29b02839bb7fb7f51&scope.project.id=57b2ea019ba3413bafb7714839f85a22&role.id=8c039234db9640d0bdc0c2a629ff9e52 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:49.289290 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbad791-7422-4da6-b913-d0fe46f32fd4 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:49.289794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbad791-7422-4da6-b913-d0fe46f32fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:49.289869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbad791-7422-4da6-b913-d0fe46f32fd4 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:49.291586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 235/456] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:49.301803 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115a796-8d32-4faf-8c66-865f00695969 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:49.302243 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115a796-8d32-4faf-8c66-865f00695969 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:49.302426 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115a796-8d32-4faf-8c66-865f00695969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:49.324435 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f115a796-8d32-4faf-8c66-865f00695969 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:49.325673 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f115a796-8d32-4faf-8c66-865f00695969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:49.342630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3c559-953c-4751-afb5-8a4f033061a4 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:49.343526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3c559-953c-4751-afb5-8a4f033061a4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:49.344013 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3c559-953c-4751-afb5-8a4f033061a4 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:49.346036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 236/457] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:49.353212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134da37b-1fd4-4ad1-986f-354d77624fe4 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:49.353806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134da37b-1fd4-4ad1-986f-354d77624fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:49.354540 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134da37b-1fd4-4ad1-986f-354d77624fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:49.390689 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-134da37b-1fd4-4ad1-986f-354d77624fe4 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:49.391908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-134da37b-1fd4-4ad1-986f-354d77624fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:49.421362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:49.427872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 222/458] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:49.456859 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115a796-8d32-4faf-8c66-865f00695969 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:49.508268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 237/459] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:49.509909 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4d58e-cc56-4415-8071-6c3caf8e7435 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:49.510615 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4d58e-cc56-4415-8071-6c3caf8e7435 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:49.511393 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4d58e-cc56-4415-8071-6c3caf8e7435 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:49.512892 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 238/460] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:49.519572 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2baac-f90e-4bde-834c-8aaf874e3e16 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:49.520046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2baac-f90e-4bde-834c-8aaf874e3e16 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:49.520597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2baac-f90e-4bde-834c-8aaf874e3e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:49.544053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-09f2baac-f90e-4bde-834c-8aaf874e3e16 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:49.545531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-09f2baac-f90e-4bde-834c-8aaf874e3e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:49.583965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115a796-8d32-4faf-8c66-865f00695969 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:49.613817 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:49.628527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 239/461] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:49.628527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2baac-f90e-4bde-834c-8aaf874e3e16 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:49.631533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b81f2-14ef-40f9-8688-49a66210b2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:49.637219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b81f2-14ef-40f9-8688-49a66210b2ef None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:49.637736 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b81f2-14ef-40f9-8688-49a66210b2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:49.638239 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b81f2-14ef-40f9-8688-49a66210b2ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:49.661199 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b81f2-14ef-40f9-8688-49a66210b2ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:49.663098 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b81f2-14ef-40f9-8688-49a66210b2ef None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:49.683619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4e1b81f2-14ef-40f9-8688-49a66210b2ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:49.683619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:49.683619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:49.683619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:49.683619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:49.683619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:49.683619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:49.683619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:49.683619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:49.683619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:49.683619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:49.683619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:49.683619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:15:49.683619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:15:49.683619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:15:49.684599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:15:49.684599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:49.684599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:49.684599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:49.684599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:49.684599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:49.684599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:49.684599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:49.684599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:49.684599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:49.684599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:49.684599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:49.684599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:49.684599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:49.684599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:49.684599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:49.684599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:49.685485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:15:49.685485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:15:49.685485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:49.685485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:49.685485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 223/462] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:15:49.690267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b81f2-14ef-40f9-8688-49a66210b2ef None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:49.736991 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2baac-f90e-4bde-834c-8aaf874e3e16 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:49.754008 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:49.776313 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bb07f-a830-4dff-b567-42d64d7f454f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:49.777140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bb07f-a830-4dff-b567-42d64d7f454f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:49.777429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bb07f-a830-4dff-b567-42d64d7f454f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:49.777704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bb07f-a830-4dff-b567-42d64d7f454f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:49.788349 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949bb07f-a830-4dff-b567-42d64d7f454f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:49.790108 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949bb07f-a830-4dff-b567-42d64d7f454f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:49.806058 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b81f2-14ef-40f9-8688-49a66210b2ef None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:49.810371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-949bb07f-a830-4dff-b567-42d64d7f454f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:49.810371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:49.810371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:49.810371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:49.810371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:49.810371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:49.810371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:49.810371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:49.810371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:49.810371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:49.810371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:49.810371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:49.810371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:15:49.810371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:15:49.810371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:15:49.811212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:15:49.811212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:49.811212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:49.811212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:49.811212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:49.811212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:49.811212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:49.811212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:49.811212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:49.811212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:49.811212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:49.811212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:49.811212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:49.811212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:49.811212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:49.811212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:49.811212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:49.812070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:15:49.812070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:15:49.812070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:49.812070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:49.812070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 240/463] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:49.818088 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f7404-46e4-445c-a26c-bf00bcbb0401 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:49.818088 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f7404-46e4-445c-a26c-bf00bcbb0401 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:49.818088 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f7404-46e4-445c-a26c-bf00bcbb0401 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:49.818758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 241/464] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:49.819603 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f7404-46e4-445c-a26c-bf00bcbb0401 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:49.846116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba20ad-12f0-4740-83e7-e11fe379f432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:49.846904 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba20ad-12f0-4740-83e7-e11fe379f432 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:49.847185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba20ad-12f0-4740-83e7-e11fe379f432 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:49.847465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba20ad-12f0-4740-83e7-e11fe379f432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:49.859464 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba20ad-12f0-4740-83e7-e11fe379f432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:49.860957 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba20ad-12f0-4740-83e7-e11fe379f432 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:49.877863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0bba20ad-12f0-4740-83e7-e11fe379f432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:49.877863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:49.877863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:49.877863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:49.877863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:49.877863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:49.877863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:49.877863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:49.877863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:49.877863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:49.877863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:49.877863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:49.877863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:15:49.877863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:15:49.877863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:15:49.878708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:15:49.878708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:49.878708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:49.878708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:49.878708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:49.878708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:49.878708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:49.878708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:49.878708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:49.878708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:49.878708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:49.878708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:49.878708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:49.878708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:49.878708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:49.878708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:49.878708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:49.879585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:15:49.879585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:15:49.879585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:49.879585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:49.879585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 224/465] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:49.880330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba20ad-12f0-4740-83e7-e11fe379f432 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:49.881194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:49.935482 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f7404-46e4-445c-a26c-bf00bcbb0401 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:49.971927 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6b9e2-1197-49c5-a2bc-c71d7e2b1bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:49.972614 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6b9e2-1197-49c5-a2bc-c71d7e2b1bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:49.972897 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6b9e2-1197-49c5-a2bc-c71d7e2b1bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:49.973169 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6b9e2-1197-49c5-a2bc-c71d7e2b1bd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:49.973887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6b9e2-1197-49c5-a2bc-c71d7e2b1bd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:49.974188 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6b9e2-1197-49c5-a2bc-c71d7e2b1bd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:49.975578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6b9e2-1197-49c5-a2bc-c71d7e2b1bd9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:49.983430 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba20ad-12f0-4740-83e7-e11fe379f432 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:49.986595 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:49.987492 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 242/466] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:49.988751 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4f07d-3837-4390-8acf-bd6241f1019f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:49.989061 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4f07d-3837-4390-8acf-bd6241f1019f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:49.989369 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4f07d-3837-4390-8acf-bd6241f1019f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:50.005743 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1fd4f07d-3837-4390-8acf-bd6241f1019f None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:50.007696 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1fd4f07d-3837-4390-8acf-bd6241f1019f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:50.026042 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670f302-19f3-4a6b-a0c5-ed5f26b67c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:50.026709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670f302-19f3-4a6b-a0c5-ed5f26b67c41 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:50.026997 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670f302-19f3-4a6b-a0c5-ed5f26b67c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:50.027278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670f302-19f3-4a6b-a0c5-ed5f26b67c41 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:50.028011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670f302-19f3-4a6b-a0c5-ed5f26b67c41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:50.028330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670f302-19f3-4a6b-a0c5-ed5f26b67c41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:50.030220 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670f302-19f3-4a6b-a0c5-ed5f26b67c41 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:50.046828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 225/467] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:50.046828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670f302-19f3-4a6b-a0c5-ed5f26b67c41 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:50.088541 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 243/468] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:50.088541 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4f07d-3837-4390-8acf-bd6241f1019f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:50.135597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:50.161061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670f302-19f3-4a6b-a0c5-ed5f26b67c41 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:50.201735 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b04be-5cb8-4953-87ea-f1454c61b05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:50.202063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4f07d-3837-4390-8acf-bd6241f1019f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:50.202321 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b04be-5cb8-4953-87ea-f1454c61b05e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:50.202605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b04be-5cb8-4953-87ea-f1454c61b05e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:50.202878 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b04be-5cb8-4953-87ea-f1454c61b05e None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:50.203764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b04be-5cb8-4953-87ea-f1454c61b05e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:50.204072 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b04be-5cb8-4953-87ea-f1454c61b05e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:50.206288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b04be-5cb8-4953-87ea-f1454c61b05e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:50.225150 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 226/469] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:50.225541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b04be-5cb8-4953-87ea-f1454c61b05e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:50.242059 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c5165-db02-47bb-8ad1-722eb5fa8ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:50.242723 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c5165-db02-47bb-8ad1-722eb5fa8ec9 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:50.243008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c5165-db02-47bb-8ad1-722eb5fa8ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:50.243286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c5165-db02-47bb-8ad1-722eb5fa8ec9 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:50.243997 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c5165-db02-47bb-8ad1-722eb5fa8ec9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:50.245472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c5165-db02-47bb-8ad1-722eb5fa8ec9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:50.252231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:50.265526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 244/470] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 18 10:15:50 2022] POST /identity/v3/users => generated 248 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:15:50.277503 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c5165-db02-47bb-8ad1-722eb5fa8ec9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:50.352568 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b04be-5cb8-4953-87ea-f1454c61b05e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:50.390825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18e3aa-1ec1-41f0-8669-69273bd2464c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:50.391507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18e3aa-1ec1-41f0-8669-69273bd2464c None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:50.391789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18e3aa-1ec1-41f0-8669-69273bd2464c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:50.392063 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18e3aa-1ec1-41f0-8669-69273bd2464c None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:50.392847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18e3aa-1ec1-41f0-8669-69273bd2464c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:50.394326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18e3aa-1ec1-41f0-8669-69273bd2464c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:50.424535 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 227/471] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 18 10:15:50 2022] POST /identity/v3/users => generated 246 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:15:50.424535 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18e3aa-1ec1-41f0-8669-69273bd2464c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:50.447547 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:50.453538 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c5165-db02-47bb-8ad1-722eb5fa8ec9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:50.507189 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2401f66-0ee7-4357-bfd0-7ddaa5701878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:50.508030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2401f66-0ee7-4357-bfd0-7ddaa5701878 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:50.508336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2401f66-0ee7-4357-bfd0-7ddaa5701878 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:50.508681 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2401f66-0ee7-4357-bfd0-7ddaa5701878 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:50.521577 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2401f66-0ee7-4357-bfd0-7ddaa5701878 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:50.523630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2401f66-0ee7-4357-bfd0-7ddaa5701878 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:50.544260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:50.549024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d2401f66-0ee7-4357-bfd0-7ddaa5701878 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:50.549024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:50.549024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:50.549024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:50.549024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:50.549024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:50.549024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:50.549024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:50.549024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:50.549024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:50.549024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:50.549024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:50.549024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:15:50.549024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:15:50.549024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:15:50.550331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:15:50.550331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:50.550331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:50.550331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:50.550331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:50.550331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:50.550331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:50.550331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:50.550331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:50.550331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:50.550331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:50.550331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:50.550331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:50.550331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:50.550331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:50.550331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:50.550331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:50.551265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:15:50.551265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:15:50.551265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:50.551265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:50.551265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 245/472] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:50.557734 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2401f66-0ee7-4357-bfd0-7ddaa5701878 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:50.581852 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18e3aa-1ec1-41f0-8669-69273bd2464c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:50.626325 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d6cb15-0e58-4bad-8874-12545658bbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:50.627002 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6cb15-0e58-4bad-8874-12545658bbc1 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:50.627283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6cb15-0e58-4bad-8874-12545658bbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:50.627561 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6cb15-0e58-4bad-8874-12545658bbc1 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:50.628273 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d6cb15-0e58-4bad-8874-12545658bbc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:50.629758 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d6cb15-0e58-4bad-8874-12545658bbc1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:50.656381 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 228/473] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 18 10:15:50 2022] POST /identity/v3/users => generated 248 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:15:50.658111 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becb9cf-f527-4348-a526-5159b7663352 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:50.658577 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becb9cf-f527-4348-a526-5159b7663352 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:50.658865 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becb9cf-f527-4348-a526-5159b7663352 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:50.660407 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 229/474] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:50.670198 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d2502-fc99-4b6b-b953-6e9750e1a1de None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:50.670490 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d2502-fc99-4b6b-b953-6e9750e1a1de None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:50.670779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d2502-fc99-4b6b-b953-6e9750e1a1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:50.690639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2401f66-0ee7-4357-bfd0-7ddaa5701878 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:50.696112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6f2d2502-fc99-4b6b-b953-6e9750e1a1de None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:50.697604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6f2d2502-fc99-4b6b-b953-6e9750e1a1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:50.732666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c2759-5541-48d1-8784-97613e36cde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:50.733409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c2759-5541-48d1-8784-97613e36cde4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:50.733880 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c2759-5541-48d1-8784-97613e36cde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:50.734172 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c2759-5541-48d1-8784-97613e36cde4 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:50.735352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c2759-5541-48d1-8784-97613e36cde4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:50.735352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c2759-5541-48d1-8784-97613e36cde4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:50.736973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c2759-5541-48d1-8784-97613e36cde4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:50.752885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 246/475] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:50.758969 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c2759-5541-48d1-8784-97613e36cde4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:50.777508 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:50.799680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 230/476] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:50.832347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d2502-fc99-4b6b-b953-6e9750e1a1de None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:50.897276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c2759-5541-48d1-8784-97613e36cde4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:50.908809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:50.937312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b0595-72ec-4ac4-ab2d-95077a883f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:50.937983 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b0595-72ec-4ac4-ab2d-95077a883f99 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:50.938271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b0595-72ec-4ac4-ab2d-95077a883f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:50.938547 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b0595-72ec-4ac4-ab2d-95077a883f99 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:50.939257 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b0595-72ec-4ac4-ab2d-95077a883f99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:50.940857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b0595-72ec-4ac4-ab2d-95077a883f99 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:50.949922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d2502-fc99-4b6b-b953-6e9750e1a1de None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:50.970555 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 247/477] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 18 10:15:50 2022] POST /identity/v3/users => generated 249 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:15:51.000231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad611469-df02-4867-88d4-bd012f73ce60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:51.001036 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad611469-df02-4867-88d4-bd012f73ce60 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:51.001397 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad611469-df02-4867-88d4-bd012f73ce60 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:51.001778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad611469-df02-4867-88d4-bd012f73ce60 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:51.002624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad611469-df02-4867-88d4-bd012f73ce60 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:51.004122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad611469-df02-4867-88d4-bd012f73ce60 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:51.012618 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.sql.core [None req-ad611469-df02-4867-88d4-bd012f73ce60 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 10:15:51.012618 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 10:15:51.012618 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [parameters: {'id': 'f84fd4c1e26f4cb8a309ed351f562c3b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 10:15:51.012618 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80142) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 10:15:51.030936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ad611469-df02-4867-88d4-bd012f73ce60 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 10:15:51.030936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:51.030936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 10:15:51.030936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 10:15:51.030936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 10:15:51.030936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 10:15:51.030936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 10:15:51.030936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 10:15:51.030936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 10:15:51.030936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application conn.query(q) Jul 18 10:15:51.030936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 10:15:51.030936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 10:15:51.030936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 10:15:51.030936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result.read() Jul 18 10:15:51.030936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 10:15:51.032093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 10:15:51.032093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 10:15:51.032093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 10:15:51.032093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 10:15:51.032093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 10:15:51.032093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 10:15:51.032093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 10:15:51.032093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 10:15:51.032093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:51.032093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 10:15:51.032093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:51.032093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:51.032093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 10:15:51.032093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 10:15:51.032093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 18 10:15:51.032093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return ref.to_dict() Jul 18 10:15:51.032093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 10:15:51.032093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application next(self.gen) Jul 18 10:15:51.033283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 10:15:51.033283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application yield resource Jul 18 10:15:51.033283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 10:15:51.033283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application next(self.gen) Jul 18 10:15:51.033283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 10:15:51.033283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 10:15:51.033283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 10:15:51.033283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 10:15:51.033283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 10:15:51.033283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise self.value Jul 18 10:15:51.033283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 10:15:51.033283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 10:15:51.033283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 10:15:51.033283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application session.commit() Jul 18 10:15:51.033283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 10:15:51.033283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 10:15:51.033283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 10:15:51.034346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 10:15:51.034346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 10:15:51.034346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.session.flush() Jul 18 10:15:51.034346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 10:15:51.034346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 10:15:51.034346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 10:15:51.034346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 10:15:51.034346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 10:15:51.034346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application compat.raise_( Jul 18 10:15:51.034346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 10:15:51.034346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception Jul 18 10:15:51.034346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 10:15:51.034346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 10:15:51.034346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 10:15:51.034346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 10:15:51.034346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 10:15:51.034346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 10:15:51.035219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 10:15:51.035219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 10:15:51.035219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 10:15:51.035219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 10:15:51.035219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 10:15:51.035219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 10:15:51.035219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 10:15:51.035219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 10:15:51.035219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 10:15:51.035219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 10:15:51.035219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 10:15:51.035219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 10:15:51.035219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 10:15:51.035219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 10:15:51.035219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 10:15:51.035219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception Jul 18 10:15:51.036075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 10:15:51.036075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 10:15:51.036075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 10:15:51.036075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 10:15:51.036075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 10:15:51.036075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 10:15:51.036075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 10:15:51.036075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application conn.query(q) Jul 18 10:15:51.036075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 10:15:51.036075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 10:15:51.036075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 10:15:51.036075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result.read() Jul 18 10:15:51.036075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 10:15:51.036075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 10:15:51.036075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 10:15:51.036075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 10:15:51.037531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 10:15:51.037531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 10:15:51.037531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 10:15:51.037531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 10:15:51.037531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 10:15:51.037531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 10:15:51.037531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [parameters: {'id': 'f84fd4c1e26f4cb8a309ed351f562c3b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 10:15:51.037531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 10:15:51.037531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:51.037531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:15:51.037531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:51.037531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:51.037531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:51.037531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:51.037531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:51.037531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:51.039057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:51.039057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:51.039057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:51.039057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:51.039057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:51.039057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:51.039057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 18 10:15:51.039057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 18 10:15:51.039057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:51.039057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:51.039057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 18 10:15:51.039057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 18 10:15:51.039057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 10:15:51.039057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 10:15:51.039057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 10:15:51.039057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:51.040112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 231/478] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Jul 18 10:15:50 2022] POST /identity/v3/roles => generated 144 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:51.040188 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b0595-72ec-4ac4-ab2d-95077a883f99 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:51.117171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:51.159754 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b0595-72ec-4ac4-ab2d-95077a883f99 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:51.196083 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81117750-e243-463c-88c4-962b4e72bc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:51.197221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81117750-e243-463c-88c4-962b4e72bc15 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:51.197658 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81117750-e243-463c-88c4-962b4e72bc15 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:51.198098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81117750-e243-463c-88c4-962b4e72bc15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:51.209595 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81117750-e243-463c-88c4-962b4e72bc15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:51.218728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-81117750-e243-463c-88c4-962b4e72bc15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:51.218728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:51.218728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:51.218728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:51.218728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:51.218728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:51.218728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:51.218728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:51.218728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:51.218728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:51.218728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:51.218728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:51.218728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:51.218728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:51.218728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:51.218728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:51.219981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:51.219981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:51.219981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:51.219981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:51.219981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:51.219981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:51.219981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:51.219981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:51.219981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:51.219981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:51.219981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:51.219981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:51.219981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:51.219981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:51.219981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:51.219981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:51.220974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:51.220974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:51.220974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:51.220974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:51.220974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:51.220974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:51.220974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:51.220974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:51.220974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:51.220974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:51.220974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 10:15:51.220974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:51.223523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 248/479] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:15:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:51.231266 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad611469-df02-4867-88d4-bd012f73ce60 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:51.359685 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad611469-df02-4867-88d4-bd012f73ce60 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:51.390266 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:51.396811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f8a5a-31c9-4bef-aae2-4f5a355d4af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:51.398267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f8a5a-31c9-4bef-aae2-4f5a355d4af5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:51.398554 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f8a5a-31c9-4bef-aae2-4f5a355d4af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:51.398869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f8a5a-31c9-4bef-aae2-4f5a355d4af5 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:51.399806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f8a5a-31c9-4bef-aae2-4f5a355d4af5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:51.400180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f8a5a-31c9-4bef-aae2-4f5a355d4af5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:51.401921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f8a5a-31c9-4bef-aae2-4f5a355d4af5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:51.410336 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 232/480] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:15:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:52.988691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3e8b5-a026-49e4-8ad0-6e7adfcda843 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:52.989200 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3e8b5-a026-49e4-8ad0-6e7adfcda843 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:52.989295 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3e8b5-a026-49e4-8ad0-6e7adfcda843 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:52.990990 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 249/481] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:52.996459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e072cc-4705-4b88-a872-cfb2a0a11c36 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:52.996879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e072cc-4705-4b88-a872-cfb2a0a11c36 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:52.997046 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e072cc-4705-4b88-a872-cfb2a0a11c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:53.019581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c689b85-36ca-4b5f-9143-376c3f70a072 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:53.019899 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c689b85-36ca-4b5f-9143-376c3f70a072 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:53.020178 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c689b85-36ca-4b5f-9143-376c3f70a072 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:53.021713 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 250/482] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:53.021888 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-43e072cc-4705-4b88-a872-cfb2a0a11c36 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:53.023620 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-43e072cc-4705-4b88-a872-cfb2a0a11c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:53.027497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cf5cf-7abd-4afb-a5f7-5b01c1e9688f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:53.027789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cf5cf-7abd-4afb-a5f7-5b01c1e9688f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:53.028073 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cf5cf-7abd-4afb-a5f7-5b01c1e9688f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:53.045585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a39cf5cf-7abd-4afb-a5f7-5b01c1e9688f None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:53.046898 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a39cf5cf-7abd-4afb-a5f7-5b01c1e9688f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:53.124624 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 251/483] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:53.145052 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39cf5cf-7abd-4afb-a5f7-5b01c1e9688f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:53.167116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 233/484] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:53.204753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:53.210783 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e072cc-4705-4b88-a872-cfb2a0a11c36 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:53.251672 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39cf5cf-7abd-4afb-a5f7-5b01c1e9688f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:53.288747 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192ad16-b3c9-45ff-b2ac-2df142973814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:53.289663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192ad16-b3c9-45ff-b2ac-2df142973814 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:53.289950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192ad16-b3c9-45ff-b2ac-2df142973814 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:53.290230 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192ad16-b3c9-45ff-b2ac-2df142973814 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:53.301324 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192ad16-b3c9-45ff-b2ac-2df142973814 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:53.310181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-5192ad16-b3c9-45ff-b2ac-2df142973814 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:15:53.310181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:53.310181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:53.310181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:53.310181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:53.310181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:53.310181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:53.310181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:53.310181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:53.310181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:53.310181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:53.310181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:53.310181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:53.310181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:53.310181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:53.310181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:53.311067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:53.311067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:53.311067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:53.311067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:53.311067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:53.311067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:53.311067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:53.311067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:53.311067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:53.311067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:53.311067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:53.311067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:53.311067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:53.311067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:53.311067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:53.311067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:53.311877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:53.311877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:53.311877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:53.311877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:53.311877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:53.311877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:53.311877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:53.311877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:53.311877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:53.311877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:53.311877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:15:53.311877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:53.316305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 252/485] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:15:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:15:53.317691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4166ca-a767-42c1-91bd-3d324fe48cd4 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:53.317986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4166ca-a767-42c1-91bd-3d324fe48cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:53.318269 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4166ca-a767-42c1-91bd-3d324fe48cd4 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:53.319998 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 253/486] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:53.325054 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4166ca-a767-42c1-91bd-3d324fe48cd4 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:53.369303 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e072cc-4705-4b88-a872-cfb2a0a11c36 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:53.394099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:53.406747 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fcf5a-5bf5-42c3-88a8-30591f4c4a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:53.407818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fcf5a-5bf5-42c3-88a8-30591f4c4a1b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:53.408377 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fcf5a-5bf5-42c3-88a8-30591f4c4a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:53.408864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fcf5a-5bf5-42c3-88a8-30591f4c4a1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:53.420099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966fcf5a-5bf5-42c3-88a8-30591f4c4a1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:53.428636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-966fcf5a-5bf5-42c3-88a8-30591f4c4a1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:15:53.428636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:53.428636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:53.428636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:53.428636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:53.428636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:53.428636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:53.428636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:53.428636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:53.428636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:53.428636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:53.428636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:53.428636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:53.428636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:53.428636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:53.428636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:53.430144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:53.430144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:53.430144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:53.430144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:53.430144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:53.430144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:53.430144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:53.430144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:53.430144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:53.430144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:53.430144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:53.430144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:53.430144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:53.430144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:53.430144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:53.430144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:53.430929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:53.430929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:53.430929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:53.430929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:53.430929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:53.430929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:53.430929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:53.430929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:53.430929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:53.430929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:53.430929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:15:53.430929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:53.433823 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 234/487] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:15:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:15:53.434058 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4166ca-a767-42c1-91bd-3d324fe48cd4 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:53.435557 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946def1e-febc-4892-80a8-33f33d58f39a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:53.436040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946def1e-febc-4892-80a8-33f33d58f39a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:53.436627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946def1e-febc-4892-80a8-33f33d58f39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:53.455615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-946def1e-febc-4892-80a8-33f33d58f39a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:53.456835 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-946def1e-febc-4892-80a8-33f33d58f39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:53.469044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:53.473656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee48a96-1eb4-43ff-8112-af92f2e84a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:53.474394 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee48a96-1eb4-43ff-8112-af92f2e84a6f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:53.475090 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee48a96-1eb4-43ff-8112-af92f2e84a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:53.475564 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee48a96-1eb4-43ff-8112-af92f2e84a6f None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:53.476950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee48a96-1eb4-43ff-8112-af92f2e84a6f None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:53.477501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee48a96-1eb4-43ff-8112-af92f2e84a6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:53.478984 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee48a96-1eb4-43ff-8112-af92f2e84a6f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:53.496596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 254/488] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:15:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:53.497953 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee48a96-1eb4-43ff-8112-af92f2e84a6f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:53.538994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 235/489] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:53.539375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946def1e-febc-4892-80a8-33f33d58f39a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:53.611506 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee48a96-1eb4-43ff-8112-af92f2e84a6f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:53.647197 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946def1e-febc-4892-80a8-33f33d58f39a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:53.651668 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae0e02-6b76-4bfa-9843-530e9ec03fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:53.652898 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae0e02-6b76-4bfa-9843-530e9ec03fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:53.653384 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae0e02-6b76-4bfa-9843-530e9ec03fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:53.653857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae0e02-6b76-4bfa-9843-530e9ec03fa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:53.654910 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae0e02-6b76-4bfa-9843-530e9ec03fa5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:53.655420 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae0e02-6b76-4bfa-9843-530e9ec03fa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:53.657163 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae0e02-6b76-4bfa-9843-530e9ec03fa5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:53.661240 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:53.669079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 255/490] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:15:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:53.670293 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae0e02-6b76-4bfa-9843-530e9ec03fa5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:53.691232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987c806-86b2-4973-b9db-6418e6dd9659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:53.692176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987c806-86b2-4973-b9db-6418e6dd9659 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:53.692415 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987c806-86b2-4973-b9db-6418e6dd9659 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:53.692729 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987c806-86b2-4973-b9db-6418e6dd9659 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:53.704143 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987c806-86b2-4973-b9db-6418e6dd9659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:53.705664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8987c806-86b2-4973-b9db-6418e6dd9659 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:53.718319 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:53.726379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8987c806-86b2-4973-b9db-6418e6dd9659 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:53.726379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:53.726379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:53.726379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:53.726379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:53.726379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:53.726379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:53.726379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:53.726379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:53.726379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:53.726379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:53.726379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:53.726379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:15:53.726379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:15:53.726379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:15:53.727649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:15:53.727649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:53.727649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:53.727649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:53.727649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:53.727649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:53.727649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:53.727649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:53.727649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:53.727649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:53.727649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:53.727649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:53.727649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:53.727649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:53.727649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:53.727649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:53.727649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:53.728767 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:15:53.728767 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:15:53.728767 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:53.728767 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:53.731617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 236/491] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:53.732615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987c806-86b2-4973-b9db-6418e6dd9659 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:53.791131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae0e02-6b76-4bfa-9843-530e9ec03fa5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:53.826339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e37ed4-faaf-4256-b268-dac72b9ddf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:53.827139 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e37ed4-faaf-4256-b268-dac72b9ddf57 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:53.827429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e37ed4-faaf-4256-b268-dac72b9ddf57 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:53.827707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e37ed4-faaf-4256-b268-dac72b9ddf57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:53.838169 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e37ed4-faaf-4256-b268-dac72b9ddf57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:53.840233 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987c806-86b2-4973-b9db-6418e6dd9659 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:53.845811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f8e37ed4-faaf-4256-b268-dac72b9ddf57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:15:53.845811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:53.845811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:53.845811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:53.845811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:53.845811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:53.845811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:53.845811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:53.845811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:53.845811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:53.845811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:53.845811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:53.845811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:53.845811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:53.845811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:53.845811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:53.846677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:53.846677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:53.846677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:53.846677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:53.846677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:53.846677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:53.846677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:53.846677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:53.846677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:53.846677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:53.846677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:53.846677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:53.846677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:53.846677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:53.846677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:53.846677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:53.847453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:53.847453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:53.847453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:53.847453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:53.847453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:53.847453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:53.847453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:53.847453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:53.847453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:53.847453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:53.847453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:15:53.847453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:53.847453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 256/492] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:15:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:53.848655 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab771056-3fd1-40ab-a986-93df91cfcbc2 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:53.848824 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab771056-3fd1-40ab-a986-93df91cfcbc2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:53.849105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab771056-3fd1-40ab-a986-93df91cfcbc2 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:53.851277 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 257/493] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:53.852004 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab771056-3fd1-40ab-a986-93df91cfcbc2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:53.881308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e8e59-02f0-4867-becd-c51f00e632be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:53.882727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e8e59-02f0-4867-becd-c51f00e632be None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:53.883015 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e8e59-02f0-4867-becd-c51f00e632be None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:53.883335 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e8e59-02f0-4867-becd-c51f00e632be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:53.894527 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e8e59-02f0-4867-becd-c51f00e632be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:53.896040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e8e59-02f0-4867-becd-c51f00e632be None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:53.914090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-214e8e59-02f0-4867-becd-c51f00e632be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:53.914090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:53.914090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:53.914090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:53.914090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:53.914090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:53.914090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:53.914090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:53.914090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:53.914090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:53.914090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:53.914090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:53.914090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:15:53.914090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:15:53.914090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:15:53.914977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:15:53.914977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:53.914977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:53.914977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:53.914977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:53.914977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:53.914977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:53.914977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:53.914977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:53.914977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:53.914977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:53.914977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:53.914977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:53.914977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:53.914977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:53.914977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:53.914977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:53.915816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:15:53.915816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:15:53.915816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:53.915816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:53.915816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 237/494] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:53.916079 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e8e59-02f0-4867-becd-c51f00e632be None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:53.935893 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:53.964751 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:53.969404 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab771056-3fd1-40ab-a986-93df91cfcbc2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:54.007104 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24adfc-7149-4bfd-a88d-01b7cd9b848e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:54.007748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24adfc-7149-4bfd-a88d-01b7cd9b848e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:54.008039 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24adfc-7149-4bfd-a88d-01b7cd9b848e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:54.008313 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24adfc-7149-4bfd-a88d-01b7cd9b848e None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:54.009081 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24adfc-7149-4bfd-a88d-01b7cd9b848e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:54.009671 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24adfc-7149-4bfd-a88d-01b7cd9b848e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:54.010767 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24adfc-7149-4bfd-a88d-01b7cd9b848e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:54.027648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e8e59-02f0-4867-becd-c51f00e632be None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:54.028114 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 258/495] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:54.028114 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103574f0-0653-4fe6-8ec9-1fa45df5c6ea None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:54.028114 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103574f0-0653-4fe6-8ec9-1fa45df5c6ea None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:54.028114 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103574f0-0653-4fe6-8ec9-1fa45df5c6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:54.044181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-103574f0-0653-4fe6-8ec9-1fa45df5c6ea None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:54.045330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-103574f0-0653-4fe6-8ec9-1fa45df5c6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:54.062696 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561aa0f-101c-4ddb-a0b3-b506e1373323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:54.063360 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561aa0f-101c-4ddb-a0b3-b506e1373323 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:54.063644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561aa0f-101c-4ddb-a0b3-b506e1373323 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:54.063921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561aa0f-101c-4ddb-a0b3-b506e1373323 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:54.064693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561aa0f-101c-4ddb-a0b3-b506e1373323 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:54.065004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561aa0f-101c-4ddb-a0b3-b506e1373323 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:54.066386 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561aa0f-101c-4ddb-a0b3-b506e1373323 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:54.074355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 238/496] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:15:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:54.075127 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561aa0f-101c-4ddb-a0b3-b506e1373323 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:54.115836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 259/497] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:54.124315 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103574f0-0653-4fe6-8ec9-1fa45df5c6ea None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:54.162100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:54.186811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561aa0f-101c-4ddb-a0b3-b506e1373323 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:54.205207 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:54.215289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103574f0-0653-4fe6-8ec9-1fa45df5c6ea None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:54.219447 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8fe9fa-a3e6-45e0-8634-4f6396c87dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:54.220099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8fe9fa-a3e6-45e0-8634-4f6396c87dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:54.220383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8fe9fa-a3e6-45e0-8634-4f6396c87dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:54.220690 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8fe9fa-a3e6-45e0-8634-4f6396c87dd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:54.221418 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8fe9fa-a3e6-45e0-8634-4f6396c87dd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:54.221722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8fe9fa-a3e6-45e0-8634-4f6396c87dd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:54.223101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8fe9fa-a3e6-45e0-8634-4f6396c87dd4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:54.235502 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 239/498] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:54.236750 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851e26c-8ff3-429a-b4a3-2371b180e5aa None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:54.237042 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851e26c-8ff3-429a-b4a3-2371b180e5aa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:54.237326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851e26c-8ff3-429a-b4a3-2371b180e5aa None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:54.238717 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 240/499] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:54.239291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851e26c-8ff3-429a-b4a3-2371b180e5aa None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:54.253773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8b128-6af1-447b-9360-7a2c3db415af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:54.254564 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8b128-6af1-447b-9360-7a2c3db415af None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:54.254849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8b128-6af1-447b-9360-7a2c3db415af None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:54.255119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8b128-6af1-447b-9360-7a2c3db415af None admin] PATH_INFO: `/v3/users/glance` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:54.262485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8b128-6af1-447b-9360-7a2c3db415af None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:54.263941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8b128-6af1-447b-9360-7a2c3db415af None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:54.280048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-73b8b128-6af1-447b-9360-7a2c3db415af None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 10:15:54.280048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:54.280048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:54.280048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:54.280048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:54.280048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:54.280048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:54.280048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:54.280048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:54.280048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:54.280048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:54.280048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:54.280048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:15:54.280048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:15:54.280048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:15:54.280048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:15:54.280988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:54.280988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:54.280988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:15:54.280988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:15:54.280988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:15:54.280988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:15:54.280988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:54.280988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:54.280988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:54.280988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:54.280988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:54.280988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:54.280988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:54.280988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:54.280988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:54.280988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:54.280988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:54.280988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:54.281876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:54.281876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:54.281876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:15:54.281876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:15:54.281876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:15:54.281876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:15:54.281876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:15:54.281876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:15:54.281876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 10:15:54.281876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:54.281876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 260/500] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:54.282432 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8b128-6af1-447b-9360-7a2c3db415af None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:54.346988 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851e26c-8ff3-429a-b4a3-2371b180e5aa None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:54.384273 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25191682-e41e-4d39-b2d7-d1675690ca2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:54.385193 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25191682-e41e-4d39-b2d7-d1675690ca2c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:54.385471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25191682-e41e-4d39-b2d7-d1675690ca2c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:54.385745 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25191682-e41e-4d39-b2d7-d1675690ca2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:54.389748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8b128-6af1-447b-9360-7a2c3db415af None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:54.392298 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:54.396222 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25191682-e41e-4d39-b2d7-d1675690ca2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:54.398483 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25191682-e41e-4d39-b2d7-d1675690ca2c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:54.416908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-25191682-e41e-4d39-b2d7-d1675690ca2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:54.416908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:54.416908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:54.416908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:54.416908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:54.416908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:54.416908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:54.416908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:54.416908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:54.416908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:54.416908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:54.416908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:54.416908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:15:54.416908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:15:54.416908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:15:54.417972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:15:54.417972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:54.417972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:54.417972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:54.417972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:54.417972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:54.417972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:54.417972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:54.417972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:54.417972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:54.417972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:54.417972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:54.417972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:54.417972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:54.417972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:54.417972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:54.417972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:54.419003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:15:54.419003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:15:54.419003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:54.419003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:54.419003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 241/501] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:54.419003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25191682-e41e-4d39-b2d7-d1675690ca2c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:54.426819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877971f-324b-4767-845f-01d6a068b6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:54.427612 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877971f-324b-4767-845f-01d6a068b6ab None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:54.427894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877971f-324b-4767-845f-01d6a068b6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:54.428169 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877971f-324b-4767-845f-01d6a068b6ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:54.439059 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8877971f-324b-4767-845f-01d6a068b6ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:54.447235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8877971f-324b-4767-845f-01d6a068b6ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:15:54.447235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:54.447235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:54.447235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:54.447235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:54.447235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:54.447235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:54.447235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:54.447235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:54.447235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:54.447235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:54.447235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:54.447235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:54.447235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:54.447235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:54.447235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:54.448308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:54.448308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:54.448308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:54.448308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:54.448308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:54.448308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:54.448308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:54.448308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:54.448308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:54.448308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:54.448308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:54.448308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:54.448308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:54.448308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:54.448308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:54.448308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:54.449370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:54.449370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:54.449370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:54.449370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:54.449370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:54.449370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:54.449370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:54.449370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:54.449370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:54.449370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:54.449370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:15:54.449370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:54.449370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 261/502] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:15:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:54.450197 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c28e05-5a3d-4a2e-b7f5-d46bef746b57 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:54.450197 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c28e05-5a3d-4a2e-b7f5-d46bef746b57 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:54.450197 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c28e05-5a3d-4a2e-b7f5-d46bef746b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:54.466779 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-10c28e05-5a3d-4a2e-b7f5-d46bef746b57 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:54.467985 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-10c28e05-5a3d-4a2e-b7f5-d46bef746b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:54.470707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:54.521638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25191682-e41e-4d39-b2d7-d1675690ca2c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:54.548528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 262/503] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:15:54.548528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c28e05-5a3d-4a2e-b7f5-d46bef746b57 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:54.559252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b929e-85df-4203-90ce-36c8f99e5f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:54.560050 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b929e-85df-4203-90ce-36c8f99e5f3b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:54.560336 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b929e-85df-4203-90ce-36c8f99e5f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:54.560656 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b929e-85df-4203-90ce-36c8f99e5f3b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:54.568062 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b929e-85df-4203-90ce-36c8f99e5f3b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:54.570121 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b929e-85df-4203-90ce-36c8f99e5f3b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:54.588133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c01b929e-85df-4203-90ce-36c8f99e5f3b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:15:54.588133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:54.588133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:54.588133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:54.588133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:54.588133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:54.588133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:54.588133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:54.588133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:54.588133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:54.588133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:54.588133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:54.588133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:15:54.588133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:15:54.588133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:15:54.588133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:15:54.589135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:54.589135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:54.589135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:15:54.589135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:15:54.589135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:15:54.589135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:15:54.589135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:54.589135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:54.589135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:54.589135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:54.589135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:54.589135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:54.589135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:54.589135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:54.589135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:54.589135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:54.589135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:54.589135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:54.590021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:54.590021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:54.590021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:15:54.590021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:15:54.590021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:15:54.590021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:15:54.590021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:15:54.590021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:15:54.590021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:15:54.590021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:54.590021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 242/504] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:15:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:54.590578 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b929e-85df-4203-90ce-36c8f99e5f3b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:54.635460 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c28e05-5a3d-4a2e-b7f5-d46bef746b57 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:54.640932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:54.669110 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3aed2b-3736-4064-a412-d7d7a548b113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:54.669780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3aed2b-3736-4064-a412-d7d7a548b113 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:54.670069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3aed2b-3736-4064-a412-d7d7a548b113 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:54.670351 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3aed2b-3736-4064-a412-d7d7a548b113 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:54.671194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3aed2b-3736-4064-a412-d7d7a548b113 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:54.671507 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3aed2b-3736-4064-a412-d7d7a548b113 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:54.673009 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3aed2b-3736-4064-a412-d7d7a548b113 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:54.689684 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b929e-85df-4203-90ce-36c8f99e5f3b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:54.697394 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:54.709348 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 263/505] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Jul 18 10:15:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:54.710692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3aed2b-3736-4064-a412-d7d7a548b113 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:54.724013 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d3ab6-e298-4a4b-a44d-1972263db51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:54.724710 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d3ab6-e298-4a4b-a44d-1972263db51a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:54.724998 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d3ab6-e298-4a4b-a44d-1972263db51a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:54.725280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d3ab6-e298-4a4b-a44d-1972263db51a None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:54.726070 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d3ab6-e298-4a4b-a44d-1972263db51a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:54.726380 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d3ab6-e298-4a4b-a44d-1972263db51a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:54.727759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d3ab6-e298-4a4b-a44d-1972263db51a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:54.739598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 243/506] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:54.740453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d3ab6-e298-4a4b-a44d-1972263db51a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:54.811874 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3aed2b-3736-4064-a412-d7d7a548b113 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:54.838538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d3ab6-e298-4a4b-a44d-1972263db51a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:54.847605 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985b82f-6422-423e-90c8-c0fe47e87162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:54.848275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985b82f-6422-423e-90c8-c0fe47e87162 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:54.848615 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985b82f-6422-423e-90c8-c0fe47e87162 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:54.848799 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:54.848896 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985b82f-6422-423e-90c8-c0fe47e87162 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:54.849633 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985b82f-6422-423e-90c8-c0fe47e87162 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:54.849966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985b82f-6422-423e-90c8-c0fe47e87162 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:54.851375 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985b82f-6422-423e-90c8-c0fe47e87162 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:54.859744 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 264/507] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:15:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:54.860552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985b82f-6422-423e-90c8-c0fe47e87162 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:54.873281 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95860e-9318-4a25-bca6-b0b25be50c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:54.874062 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95860e-9318-4a25-bca6-b0b25be50c35 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:54.874610 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95860e-9318-4a25-bca6-b0b25be50c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:54.874610 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95860e-9318-4a25-bca6-b0b25be50c35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:54.885088 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95860e-9318-4a25-bca6-b0b25be50c35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:54.886621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95860e-9318-4a25-bca6-b0b25be50c35 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:54.903605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ea95860e-9318-4a25-bca6-b0b25be50c35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:54.903605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:54.903605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:54.903605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:54.903605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:54.903605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:54.903605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:54.903605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:54.903605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:54.903605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:54.903605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:54.903605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:54.903605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:15:54.903605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:15:54.903605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:15:54.904549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:15:54.904549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:54.904549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:54.904549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:54.904549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:54.904549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:54.904549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:54.904549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:54.904549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:54.904549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:54.904549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:54.904549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:54.904549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:54.904549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:54.904549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:54.904549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:54.904549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:54.905465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:15:54.905465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:15:54.905465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:54.905465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:54.905465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 244/508] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:54.905888 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95860e-9318-4a25-bca6-b0b25be50c35 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:54.925405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:54.966029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985b82f-6422-423e-90c8-c0fe47e87162 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:54.999305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4d53d-aede-4c8e-9882-2dea39c8960f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:54.999963 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4d53d-aede-4c8e-9882-2dea39c8960f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:55.000244 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4d53d-aede-4c8e-9882-2dea39c8960f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:55.000536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4d53d-aede-4c8e-9882-2dea39c8960f None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:55.001375 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc4d53d-aede-4c8e-9882-2dea39c8960f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:55.001683 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc4d53d-aede-4c8e-9882-2dea39c8960f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:55.003127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc4d53d-aede-4c8e-9882-2dea39c8960f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:55.004234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95860e-9318-4a25-bca6-b0b25be50c35 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:55.021442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 265/509] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Jul 18 10:15:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:55.022251 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4d53d-aede-4c8e-9882-2dea39c8960f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:55.037594 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fa952-555b-45d3-a8b8-eb448552b9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:55.038268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fa952-555b-45d3-a8b8-eb448552b9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:55.038552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fa952-555b-45d3-a8b8-eb448552b9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:55.038831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fa952-555b-45d3-a8b8-eb448552b9c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:55.068610 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:55.106684 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6fa952-555b-45d3-a8b8-eb448552b9c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:55.108147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6fa952-555b-45d3-a8b8-eb448552b9c2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:55.119889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4d53d-aede-4c8e-9882-2dea39c8960f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:55.137058 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:55.154274 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e168e06-a769-42b8-be9b-ebfe54d90ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:55.155063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e168e06-a769-42b8-be9b-ebfe54d90ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:55.155867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e168e06-a769-42b8-be9b-ebfe54d90ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:55.155867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e168e06-a769-42b8-be9b-ebfe54d90ad0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:55.162947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e168e06-a769-42b8-be9b-ebfe54d90ad0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:55.165151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e168e06-a769-42b8-be9b-ebfe54d90ad0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:55.179867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-1e168e06-a769-42b8-be9b-ebfe54d90ad0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:15:55.179867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:55.179867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:55.179867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:55.179867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:55.179867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:55.179867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:55.179867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:55.179867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:55.179867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:55.179867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:55.179867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:55.179867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:15:55.179867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:15:55.179867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:15:55.179867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:15:55.181022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:55.181022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:55.181022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:15:55.181022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:15:55.181022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:15:55.181022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:15:55.181022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:55.181022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:55.181022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:55.181022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:55.181022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:55.181022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:55.181022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:55.181022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:55.181022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:55.181022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:55.181022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:55.181022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:55.182132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:55.182132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:55.182132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:15:55.182132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:15:55.182132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:15:55.182132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:15:55.182132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:15:55.182132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:15:55.182132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:15:55.182132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:55.182132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 266/510] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jul 18 10:15:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:55.182686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e168e06-a769-42b8-be9b-ebfe54d90ad0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:55.212554 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 245/511] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:55.213395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fa952-555b-45d3-a8b8-eb448552b9c2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:55.274335 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:55.281299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e168e06-a769-42b8-be9b-ebfe54d90ad0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:55.314201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fa952-555b-45d3-a8b8-eb448552b9c2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:55.315965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2291d-8b22-4828-a950-6081e407c5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:55.316788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2291d-8b22-4828-a950-6081e407c5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:55.317098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2291d-8b22-4828-a950-6081e407c5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:55.317374 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2291d-8b22-4828-a950-6081e407c5f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:55.327780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2291d-8b22-4828-a950-6081e407c5f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:55.329287 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2291d-8b22-4828-a950-6081e407c5f8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:55.346778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8cc2291d-8b22-4828-a950-6081e407c5f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:55.346778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:55.346778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:55.346778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:55.346778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:55.346778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:55.346778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:55.346778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:55.346778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:55.346778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:55.346778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:55.346778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:55.346778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:15:55.346778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:15:55.346778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:15:55.347836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:15:55.347836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:55.347836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:55.347836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:55.347836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:55.347836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:55.347836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:55.347836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:55.347836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:55.347836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:55.347836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:55.347836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:55.347836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:55.347836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:55.347836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:55.347836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:55.347836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:55.348765 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:15:55.348765 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:15:55.348765 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:55.348765 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:55.348765 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 267/512] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:55.349239 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2291d-8b22-4828-a950-6081e407c5f8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:55.351091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2747e17-b10d-4b0d-87b8-3fa11f1e1412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:55.351759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2747e17-b10d-4b0d-87b8-3fa11f1e1412 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:55.352240 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2747e17-b10d-4b0d-87b8-3fa11f1e1412 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:55.352576 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2747e17-b10d-4b0d-87b8-3fa11f1e1412 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:55.353674 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2747e17-b10d-4b0d-87b8-3fa11f1e1412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:55.353982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2747e17-b10d-4b0d-87b8-3fa11f1e1412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:55.355604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2747e17-b10d-4b0d-87b8-3fa11f1e1412 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:55.367365 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 246/513] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:55.368156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2747e17-b10d-4b0d-87b8-3fa11f1e1412 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:55.382605 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:55.453720 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2291d-8b22-4828-a950-6081e407c5f8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:55.465459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2747e17-b10d-4b0d-87b8-3fa11f1e1412 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:55.486993 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73c7fe-02cb-428c-bb02-4334f660205c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:55.487497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:55.487631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73c7fe-02cb-428c-bb02-4334f660205c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:55.487911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73c7fe-02cb-428c-bb02-4334f660205c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:55.488213 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73c7fe-02cb-428c-bb02-4334f660205c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:55.499844 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8646ee1-eead-47ed-bb83-6eb6d5a4d8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:55.500528 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8646ee1-eead-47ed-bb83-6eb6d5a4d8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:55.500816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8646ee1-eead-47ed-bb83-6eb6d5a4d8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:55.501093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8646ee1-eead-47ed-bb83-6eb6d5a4d8b7 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:55.501926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8646ee1-eead-47ed-bb83-6eb6d5a4d8b7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:55.502236 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8646ee1-eead-47ed-bb83-6eb6d5a4d8b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:55.503672 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8646ee1-eead-47ed-bb83-6eb6d5a4d8b7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:55.540043 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 247/514] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jul 18 10:15:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:55.540890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8646ee1-eead-47ed-bb83-6eb6d5a4d8b7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:55.556518 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73c7fe-02cb-428c-bb02-4334f660205c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:55.558016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73c7fe-02cb-428c-bb02-4334f660205c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:55.586757 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:55.636648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8646ee1-eead-47ed-bb83-6eb6d5a4d8b7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:55.658003 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 268/515] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:55.658999 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73c7fe-02cb-428c-bb02-4334f660205c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:55.675713 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c97b8-674d-4fa5-8307-7974bec86f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:55.676497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c97b8-674d-4fa5-8307-7974bec86f1e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:55.676808 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c97b8-674d-4fa5-8307-7974bec86f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:55.677105 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c97b8-674d-4fa5-8307-7974bec86f1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:55.686969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c97b8-674d-4fa5-8307-7974bec86f1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:55.688430 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c97b8-674d-4fa5-8307-7974bec86f1e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:55.705408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c11c97b8-674d-4fa5-8307-7974bec86f1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:55.705408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:55.705408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:55.705408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:55.705408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:55.705408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:55.705408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:55.705408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:55.705408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:55.705408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:55.705408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:55.705408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:55.705408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:15:55.705408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:15:55.705408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:15:55.706208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:15:55.706208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:55.706208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:55.706208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:55.706208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:55.706208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:55.706208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:55.706208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:55.706208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:55.706208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:55.706208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:55.706208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:55.706208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:55.706208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:55.706208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:55.706208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:55.706208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:55.707039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:15:55.707039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:15:55.707039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:55.707039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:55.707039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 248/516] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:55.707673 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c97b8-674d-4fa5-8307-7974bec86f1e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:55.738988 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:55.758270 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73c7fe-02cb-428c-bb02-4334f660205c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:55.786989 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:55.794049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50028bc8-fed3-40bb-9e24-38062437e8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:55.794721 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50028bc8-fed3-40bb-9e24-38062437e8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:55.795002 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50028bc8-fed3-40bb-9e24-38062437e8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:55.795283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50028bc8-fed3-40bb-9e24-38062437e8a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:55.795999 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50028bc8-fed3-40bb-9e24-38062437e8a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:55.796303 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50028bc8-fed3-40bb-9e24-38062437e8a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:55.797717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50028bc8-fed3-40bb-9e24-38062437e8a5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:55.806707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c97b8-674d-4fa5-8307-7974bec86f1e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:55.808841 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 269/517] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:55.809621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50028bc8-fed3-40bb-9e24-38062437e8a5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:55.843117 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6406c-287c-4b51-92aa-e643479f9445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:55.843777 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6406c-287c-4b51-92aa-e643479f9445 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:55.844066 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6406c-287c-4b51-92aa-e643479f9445 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:55.844345 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6406c-287c-4b51-92aa-e643479f9445 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:55.845076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6406c-287c-4b51-92aa-e643479f9445 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:55.846518 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6406c-287c-4b51-92aa-e643479f9445 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:55.866955 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 249/518] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 18 10:15:55 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:15:55.867969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6406c-287c-4b51-92aa-e643479f9445 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:55.913870 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50028bc8-fed3-40bb-9e24-38062437e8a5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:55.951496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0833f-b7a4-42ca-bd15-72ca6d7efcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:55.952161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0833f-b7a4-42ca-bd15-72ca6d7efcda None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:55.952447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0833f-b7a4-42ca-bd15-72ca6d7efcda None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:55.952760 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0833f-b7a4-42ca-bd15-72ca6d7efcda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:55.994837 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:56.019310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6406c-287c-4b51-92aa-e643479f9445 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:56.022032 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:56.037810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0833f-b7a4-42ca-bd15-72ca6d7efcda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:56.039301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0833f-b7a4-42ca-bd15-72ca6d7efcda None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:56.064009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691855bc-90d0-4d0d-acc4-d94cf0719d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:56.064863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691855bc-90d0-4d0d-acc4-d94cf0719d0b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:56.065145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691855bc-90d0-4d0d-acc4-d94cf0719d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:56.065571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691855bc-90d0-4d0d-acc4-d94cf0719d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:56.077057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691855bc-90d0-4d0d-acc4-d94cf0719d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:56.078556 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691855bc-90d0-4d0d-acc4-d94cf0719d0b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:56.101931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-691855bc-90d0-4d0d-acc4-d94cf0719d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:56.101931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:56.101931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:56.101931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:56.101931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:56.101931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:56.101931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:56.101931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:56.101931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:56.101931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:56.101931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:56.101931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:56.101931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:15:56.101931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:15:56.101931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:15:56.102839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:15:56.102839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:56.102839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:56.102839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:56.102839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:56.102839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:56.102839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:56.102839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:56.102839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:56.102839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:56.102839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:56.102839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:56.102839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:56.102839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:56.102839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:56.102839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:56.102839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:56.103694 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:15:56.103694 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:15:56.103694 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:56.103694 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:56.103694 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 250/519] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:56.104948 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691855bc-90d0-4d0d-acc4-d94cf0719d0b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:56.153574 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 270/520] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:56.153574 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0833f-b7a4-42ca-bd15-72ca6d7efcda None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:56.219060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691855bc-90d0-4d0d-acc4-d94cf0719d0b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:56.250904 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:56.255909 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200834a-e60f-408b-9898-e89aa236832f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:56.256635 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200834a-e60f-408b-9898-e89aa236832f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:56.257556 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200834a-e60f-408b-9898-e89aa236832f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:56.257702 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200834a-e60f-408b-9898-e89aa236832f None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:56.258437 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1200834a-e60f-408b-9898-e89aa236832f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:56.258742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1200834a-e60f-408b-9898-e89aa236832f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:56.260214 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1200834a-e60f-408b-9898-e89aa236832f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:56.263028 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0833f-b7a4-42ca-bd15-72ca6d7efcda None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:56.269680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:56.274792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 251/521] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:56.279687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200834a-e60f-408b-9898-e89aa236832f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:56.299799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79fa4f-d886-4b51-b972-aa6eed4ea705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:56.300635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79fa4f-d886-4b51-b972-aa6eed4ea705 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:56.300928 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79fa4f-d886-4b51-b972-aa6eed4ea705 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:56.301193 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79fa4f-d886-4b51-b972-aa6eed4ea705 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:56.309089 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba79fa4f-d886-4b51-b972-aa6eed4ea705 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:56.310540 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba79fa4f-d886-4b51-b972-aa6eed4ea705 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:56.325155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ba79fa4f-d886-4b51-b972-aa6eed4ea705 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 10:15:56.325155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:56.325155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:56.325155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:56.325155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:56.325155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:56.325155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:56.325155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:56.325155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:56.325155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:56.325155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:56.325155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:56.325155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:15:56.325155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:15:56.325155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:15:56.325155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:15:56.326093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:56.326093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:56.326093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:15:56.326093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:15:56.326093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:15:56.326093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:15:56.326093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:56.326093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:56.326093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:56.326093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:56.326093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:56.326093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:56.326093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:56.326093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:56.326093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:56.326093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:56.326093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:56.326093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:56.327030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:56.327030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:56.327030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:15:56.327030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:15:56.327030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:15:56.327030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:15:56.327030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:15:56.327030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:15:56.327030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 10:15:56.327030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:56.327030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 271/522] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:15:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:56.328222 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79fa4f-d886-4b51-b972-aa6eed4ea705 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:56.403612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200834a-e60f-408b-9898-e89aa236832f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:56.439636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50c79e-40aa-44e9-9738-82d23fb0cee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:56.440296 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50c79e-40aa-44e9-9738-82d23fb0cee0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:56.440626 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50c79e-40aa-44e9-9738-82d23fb0cee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:56.440907 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50c79e-40aa-44e9-9738-82d23fb0cee0 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:56.441132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79fa4f-d886-4b51-b972-aa6eed4ea705 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:56.441629 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50c79e-40aa-44e9-9738-82d23fb0cee0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:56.441936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50c79e-40aa-44e9-9738-82d23fb0cee0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:56.443317 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50c79e-40aa-44e9-9738-82d23fb0cee0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:56.460627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 252/523] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:56.460627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50c79e-40aa-44e9-9738-82d23fb0cee0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:56.484708 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ed5bc-e24f-4a4f-8b04-fbb876d66c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:56.485573 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ed5bc-e24f-4a4f-8b04-fbb876d66c7d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:56.485918 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ed5bc-e24f-4a4f-8b04-fbb876d66c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:56.486147 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ed5bc-e24f-4a4f-8b04-fbb876d66c7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:56.497483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ed5bc-e24f-4a4f-8b04-fbb876d66c7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:56.498933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ed5bc-e24f-4a4f-8b04-fbb876d66c7d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:56.518055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:56.519221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-850ed5bc-e24f-4a4f-8b04-fbb876d66c7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:56.519221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:56.519221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:56.519221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:56.519221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:56.519221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:56.519221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:56.519221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:56.519221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:56.519221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:56.519221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:56.519221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:56.519221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:15:56.519221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:15:56.519221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:15:56.520457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:15:56.520457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:56.520457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:56.520457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:56.520457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:56.520457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:56.520457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:56.520457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:56.520457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:56.520457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:56.520457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:56.520457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:56.520457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:56.520457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:56.520457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:56.520457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:56.520457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:56.521663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:15:56.521663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:15:56.521663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:56.521663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:56.521663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 272/524] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:56.522298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ed5bc-e24f-4a4f-8b04-fbb876d66c7d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:56.525328 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:56.569022 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50c79e-40aa-44e9-9738-82d23fb0cee0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:56.608930 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05072745-8d7e-4464-a3a4-afa7c463aaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:56.609728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05072745-8d7e-4464-a3a4-afa7c463aaae None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:56.610017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05072745-8d7e-4464-a3a4-afa7c463aaae None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:56.610291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05072745-8d7e-4464-a3a4-afa7c463aaae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:56.617383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05072745-8d7e-4464-a3a4-afa7c463aaae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:56.618846 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05072745-8d7e-4464-a3a4-afa7c463aaae None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:56.633652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-05072745-8d7e-4464-a3a4-afa7c463aaae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:15:56.633652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:56.633652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:56.633652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:56.633652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:56.633652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:56.633652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:56.633652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:56.633652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:56.633652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:56.633652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:56.633652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:56.633652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:15:56.633652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:15:56.633652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:15:56.634545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:15:56.634545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:56.634545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:56.634545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:56.634545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:56.634545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:56.634545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:56.634545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:56.634545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:56.634545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:56.634545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:56.634545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:56.634545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:56.634545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:56.634545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:56.634545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:56.634545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:56.635395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:15:56.635395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:15:56.635395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:15:56.635395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:15:56.635395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:15:56.635395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:15:56.635395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:15:56.635395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:56.635395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 253/525] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:15:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:56.635960 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05072745-8d7e-4464-a3a4-afa7c463aaae None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:56.637345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ed5bc-e24f-4a4f-8b04-fbb876d66c7d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:56.676419 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78234f8d-eac0-48da-8aa1-afa0ac2fcab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:56.677176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78234f8d-eac0-48da-8aa1-afa0ac2fcab3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:56.677465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78234f8d-eac0-48da-8aa1-afa0ac2fcab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:56.677745 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78234f8d-eac0-48da-8aa1-afa0ac2fcab3 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:56.678589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78234f8d-eac0-48da-8aa1-afa0ac2fcab3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:56.679440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78234f8d-eac0-48da-8aa1-afa0ac2fcab3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:56.680917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78234f8d-eac0-48da-8aa1-afa0ac2fcab3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:56.699045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 273/526] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Jul 18 10:15:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:56.699045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78234f8d-eac0-48da-8aa1-afa0ac2fcab3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:56.749747 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05072745-8d7e-4464-a3a4-afa7c463aaae None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:56.765512 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:56.771106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:56.787257 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a209ef8-2d9d-44bc-a937-babda9ddf6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:56.788900 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a209ef8-2d9d-44bc-a937-babda9ddf6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:56.789438 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a209ef8-2d9d-44bc-a937-babda9ddf6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:56.789894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a209ef8-2d9d-44bc-a937-babda9ddf6a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:56.797568 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a209ef8-2d9d-44bc-a937-babda9ddf6a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:56.799252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a209ef8-2d9d-44bc-a937-babda9ddf6a2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:56.809798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78234f8d-eac0-48da-8aa1-afa0ac2fcab3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:56.814542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9a209ef8-2d9d-44bc-a937-babda9ddf6a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:15:56.814542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:56.814542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:56.814542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:56.814542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:56.814542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:56.814542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:56.814542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:56.814542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:56.814542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:56.814542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:56.814542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:56.814542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:15:56.814542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:15:56.814542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:15:56.815583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:15:56.815583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:56.815583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:56.815583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:56.815583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:56.815583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:56.815583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:56.815583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:56.815583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:56.815583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:56.815583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:56.815583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:56.815583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:56.815583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:56.815583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:56.815583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:56.815583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:56.816680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:15:56.816680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:15:56.816680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:15:56.816680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:15:56.816680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:15:56.816680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:15:56.816680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:15:56.816680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:56.818482 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 254/527] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:15:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:56.819993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a209ef8-2d9d-44bc-a937-babda9ddf6a2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:56.850364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9e987-8c2a-4971-8040-b4ba0e5d75ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:56.851053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9e987-8c2a-4971-8040-b4ba0e5d75ea None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:56.851341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9e987-8c2a-4971-8040-b4ba0e5d75ea None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:56.851620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9e987-8c2a-4971-8040-b4ba0e5d75ea None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:56.852345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9e987-8c2a-4971-8040-b4ba0e5d75ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:56.852689 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9e987-8c2a-4971-8040-b4ba0e5d75ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:56.854062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9e987-8c2a-4971-8040-b4ba0e5d75ea None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:56.866657 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 274/528] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:56.867566 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9e987-8c2a-4971-8040-b4ba0e5d75ea None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:56.932622 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a209ef8-2d9d-44bc-a937-babda9ddf6a2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:56.970830 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4629ded-13b5-4554-9539-ca10378b8874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:56.971494 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4629ded-13b5-4554-9539-ca10378b8874 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:56.971779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4629ded-13b5-4554-9539-ca10378b8874 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:56.972058 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4629ded-13b5-4554-9539-ca10378b8874 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:56.972878 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4629ded-13b5-4554-9539-ca10378b8874 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:56.973191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4629ded-13b5-4554-9539-ca10378b8874 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:56.973392 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9e987-8c2a-4971-8040-b4ba0e5d75ea None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:56.974622 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4629ded-13b5-4554-9539-ca10378b8874 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:56.992834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 255/529] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:15:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:56.993660 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4629ded-13b5-4554-9539-ca10378b8874 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:57.002709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:57.023537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b7a7c-0442-4f68-bfee-ed71ddaf008e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:57.024445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b7a7c-0442-4f68-bfee-ed71ddaf008e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:57.024576 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b7a7c-0442-4f68-bfee-ed71ddaf008e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:57.024839 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b7a7c-0442-4f68-bfee-ed71ddaf008e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:57.040366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:57.096023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b7a7c-0442-4f68-bfee-ed71ddaf008e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:57.097509 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b7a7c-0442-4f68-bfee-ed71ddaf008e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:57.103413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4629ded-13b5-4554-9539-ca10378b8874 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:57.140181 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6cd0c-3b72-43e3-9506-39924b383753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:57.140878 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6cd0c-3b72-43e3-9506-39924b383753 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:57.141171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6cd0c-3b72-43e3-9506-39924b383753 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:57.141446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6cd0c-3b72-43e3-9506-39924b383753 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:57.142244 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6cd0c-3b72-43e3-9506-39924b383753 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:57.142561 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6cd0c-3b72-43e3-9506-39924b383753 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:57.144010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6cd0c-3b72-43e3-9506-39924b383753 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:57.159330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 256/530] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:15:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:57.160250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6cd0c-3b72-43e3-9506-39924b383753 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:57.212927 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 275/531] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:15:57.217904 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b7a7c-0442-4f68-bfee-ed71ddaf008e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:57.239748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:57.275895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:57.279598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6cd0c-3b72-43e3-9506-39924b383753 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:57.316404 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48270c2a-b8ca-4eaa-9157-2dc410ba4d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:57.317232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48270c2a-b8ca-4eaa-9157-2dc410ba4d50 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:57.317526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48270c2a-b8ca-4eaa-9157-2dc410ba4d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:57.317808 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48270c2a-b8ca-4eaa-9157-2dc410ba4d50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:57.318835 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b7a7c-0442-4f68-bfee-ed71ddaf008e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:57.325002 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48270c2a-b8ca-4eaa-9157-2dc410ba4d50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:57.326462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48270c2a-b8ca-4eaa-9157-2dc410ba4d50 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:57.342440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-48270c2a-b8ca-4eaa-9157-2dc410ba4d50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:15:57.342440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:57.342440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:57.342440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:57.342440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:57.342440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:57.342440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:57.342440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:57.342440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:57.342440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:57.342440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:57.342440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:57.342440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:15:57.342440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:15:57.342440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:15:57.343314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:15:57.343314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:57.343314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:57.343314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:57.343314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:57.343314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:57.343314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:57.343314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:57.343314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:57.343314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:57.343314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:57.343314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:57.343314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:57.343314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:57.343314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:57.343314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:57.343314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:57.344161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:15:57.344161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:15:57.344161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:15:57.344161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:15:57.344161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:15:57.344161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:15:57.344161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:15:57.344161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:57.344161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 257/532] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:15:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:57.345192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48270c2a-b8ca-4eaa-9157-2dc410ba4d50 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:57.359935 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb69806-67fb-4f69-bae9-3f99b6f6eafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:57.360637 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb69806-67fb-4f69-bae9-3f99b6f6eafa None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:57.360914 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb69806-67fb-4f69-bae9-3f99b6f6eafa None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:57.361193 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb69806-67fb-4f69-bae9-3f99b6f6eafa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:57.362033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb69806-67fb-4f69-bae9-3f99b6f6eafa None admin] RBAC: Adding query filter params (role.id=f57b3e871ce142a090d6aa7dea9ac7d5, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=3cee4d23797e430e9eeddc394ed7f50e) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:57.362348 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb69806-67fb-4f69-bae9-3f99b6f6eafa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:57.363815 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb69806-67fb-4f69-bae9-3f99b6f6eafa None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:57.376565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 276/533] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:57 2022] GET /identity/v3/role_assignments?user.id=3cee4d23797e430e9eeddc394ed7f50e&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=f57b3e871ce142a090d6aa7dea9ac7d5 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:57.376565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb69806-67fb-4f69-bae9-3f99b6f6eafa None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:57.463893 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48270c2a-b8ca-4eaa-9157-2dc410ba4d50 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:57.483491 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:57.510717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb69806-67fb-4f69-bae9-3f99b6f6eafa None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:57.524221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90a4cf-f7ba-41de-8552-437dd55dcef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:57.525345 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90a4cf-f7ba-41de-8552-437dd55dcef9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:57.525858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90a4cf-f7ba-41de-8552-437dd55dcef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:57.526407 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90a4cf-f7ba-41de-8552-437dd55dcef9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:57.527521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f90a4cf-f7ba-41de-8552-437dd55dcef9 None admin] RBAC: Adding query filter params (role.id=f57b3e871ce142a090d6aa7dea9ac7d5, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=20e29c37dc924514a882f936f5936ae1) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:57.528617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f90a4cf-f7ba-41de-8552-437dd55dcef9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:57.530708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f90a4cf-f7ba-41de-8552-437dd55dcef9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:57.540247 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 258/534] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:57 2022] GET /identity/v3/role_assignments?user.id=20e29c37dc924514a882f936f5936ae1&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=f57b3e871ce142a090d6aa7dea9ac7d5 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:57.541437 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90a4cf-f7ba-41de-8552-437dd55dcef9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:57.574495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:57.596925 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b96cee2-9cb1-40e6-8bcd-b7c664c1592b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:57.597827 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96cee2-9cb1-40e6-8bcd-b7c664c1592b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:57.598204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96cee2-9cb1-40e6-8bcd-b7c664c1592b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:57.598531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96cee2-9cb1-40e6-8bcd-b7c664c1592b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:57.633965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b96cee2-9cb1-40e6-8bcd-b7c664c1592b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:57.635059 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b96cee2-9cb1-40e6-8bcd-b7c664c1592b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:57.661964 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90a4cf-f7ba-41de-8552-437dd55dcef9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:57.670202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-6b96cee2-9cb1-40e6-8bcd-b7c664c1592b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:57.670202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:57.670202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:57.670202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:57.670202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:57.670202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:57.670202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:57.670202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:57.670202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:57.670202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:57.670202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:57.670202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:57.670202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:15:57.670202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:15:57.670202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:15:57.671069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:15:57.671069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:57.671069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:57.671069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:57.671069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:57.671069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:57.671069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:57.671069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:57.671069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:57.671069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:57.671069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:57.671069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:57.671069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:57.671069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:57.671069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:57.671069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:57.671069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:57.671957 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:15:57.671957 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:15:57.671957 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:57.671957 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:57.681054 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 277/535] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:57.686794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b96cee2-9cb1-40e6-8bcd-b7c664c1592b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:57.703108 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933fbc57-103f-43c7-9976-ced7e416cdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:57.704154 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fbc57-103f-43c7-9976-ced7e416cdb9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:57.704683 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fbc57-103f-43c7-9976-ced7e416cdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:57.705193 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fbc57-103f-43c7-9976-ced7e416cdb9 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:57.706202 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933fbc57-103f-43c7-9976-ced7e416cdb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:57.706739 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933fbc57-103f-43c7-9976-ced7e416cdb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:57.708434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933fbc57-103f-43c7-9976-ced7e416cdb9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:57.723176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 259/536] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:15:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:57.730117 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933fbc57-103f-43c7-9976-ced7e416cdb9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:57.870660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b96cee2-9cb1-40e6-8bcd-b7c664c1592b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:57.909487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365694b-ae8a-4815-be81-8174af8d8ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:57.910498 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365694b-ae8a-4815-be81-8174af8d8ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:57.910986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365694b-ae8a-4815-be81-8174af8d8ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:57.911479 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365694b-ae8a-4815-be81-8174af8d8ab5 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:57.912423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e365694b-ae8a-4815-be81-8174af8d8ab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:57.912944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e365694b-ae8a-4815-be81-8174af8d8ab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:57.914545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e365694b-ae8a-4815-be81-8174af8d8ab5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:57.924868 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:57.929966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 278/537] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:57.937646 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365694b-ae8a-4815-be81-8174af8d8ab5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:57.940846 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:57.959969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933fbc57-103f-43c7-9976-ced7e416cdb9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:57.996887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60efd2-636e-401e-a477-e5198cf25013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:57.997628 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60efd2-636e-401e-a477-e5198cf25013 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:57.997858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60efd2-636e-401e-a477-e5198cf25013 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:57.998095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60efd2-636e-401e-a477-e5198cf25013 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:57.998932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60efd2-636e-401e-a477-e5198cf25013 None admin] RBAC: Adding query filter params (role.id=f57b3e871ce142a090d6aa7dea9ac7d5, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=29048c2021e340b081ae44a390583e7a) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:57.999246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60efd2-636e-401e-a477-e5198cf25013 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:58.000740 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60efd2-636e-401e-a477-e5198cf25013 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:58.006676 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 260/538] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:57 2022] GET /identity/v3/role_assignments?user.id=29048c2021e340b081ae44a390583e7a&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=f57b3e871ce142a090d6aa7dea9ac7d5 => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:58.008478 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4a53b-affa-482c-bcb6-b85bd63c10ad None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:58.015753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4a53b-affa-482c-bcb6-b85bd63c10ad None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:58.016059 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4a53b-affa-482c-bcb6-b85bd63c10ad None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:58.017855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 261/539] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:58 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:58.023353 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd48298-f5c4-4775-81f5-7d2a2adfe0b2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:58.023657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd48298-f5c4-4775-81f5-7d2a2adfe0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:58.023909 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd48298-f5c4-4775-81f5-7d2a2adfe0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:58.049236 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8cd48298-f5c4-4775-81f5-7d2a2adfe0b2 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:58.050676 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8cd48298-f5c4-4775-81f5-7d2a2adfe0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:58.068354 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365694b-ae8a-4815-be81-8174af8d8ab5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:58.109168 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f25a7f-08b3-46f6-b068-7dd271270a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:58.110354 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f25a7f-08b3-46f6-b068-7dd271270a4d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:58.110836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f25a7f-08b3-46f6-b068-7dd271270a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:58.111322 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f25a7f-08b3-46f6-b068-7dd271270a4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:58.120559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f25a7f-08b3-46f6-b068-7dd271270a4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:58.122577 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f25a7f-08b3-46f6-b068-7dd271270a4d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:58.140137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c2f25a7f-08b3-46f6-b068-7dd271270a4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:15:58.140137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:58.140137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:58.140137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:58.140137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:58.140137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:58.140137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:58.140137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:58.140137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:58.140137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:58.140137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:58.140137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:58.140137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:15:58.140137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:15:58.140137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:15:58.141079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:15:58.141079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:58.141079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:58.141079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:58.141079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:58.141079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:58.141079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:58.141079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:58.141079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:58.141079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:58.141079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:58.141079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:58.141079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:58.141079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:58.141079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:58.141079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:58.141079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:58.141906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:15:58.141906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:15:58.141906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:15:58.141906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:15:58.141906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:15:58.141906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:15:58.141906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:15:58.141906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:58.144538 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 279/540] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:15:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:58.151237 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f25a7f-08b3-46f6-b068-7dd271270a4d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:58.240447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:58.243955 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 262/541] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:58.287746 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f25a7f-08b3-46f6-b068-7dd271270a4d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:58.290046 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd48298-f5c4-4775-81f5-7d2a2adfe0b2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:58.359708 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4880286-5f36-4707-a6e7-f496c1601612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:58.360375 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4880286-5f36-4707-a6e7-f496c1601612 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:58.360683 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4880286-5f36-4707-a6e7-f496c1601612 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:58.360958 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4880286-5f36-4707-a6e7-f496c1601612 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:58.362132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4880286-5f36-4707-a6e7-f496c1601612 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:58.362447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4880286-5f36-4707-a6e7-f496c1601612 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:58.363963 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4880286-5f36-4707-a6e7-f496c1601612 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:58.402717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 280/542] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:15:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:58.407273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4880286-5f36-4707-a6e7-f496c1601612 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:58.424887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:58.458652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd48298-f5c4-4775-81f5-7d2a2adfe0b2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:58.497461 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fdcf8-da18-4cf0-81ba-8d9e91a247a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:58.498302 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fdcf8-da18-4cf0-81ba-8d9e91a247a7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:58.498596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fdcf8-da18-4cf0-81ba-8d9e91a247a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:58.498873 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fdcf8-da18-4cf0-81ba-8d9e91a247a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:58.512207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6fdcf8-da18-4cf0-81ba-8d9e91a247a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:15:58.525867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ce6fdcf8-da18-4cf0-81ba-8d9e91a247a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:15:58.525867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:58.525867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:58.525867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:58.525867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:58.525867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:58.525867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:58.525867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:58.525867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:58.525867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:58.525867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:58.525867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:58.525867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:15:58.525867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:15:58.525867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:15:58.525867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:15:58.526789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:15:58.526789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:15:58.526789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:15:58.526789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:15:58.526789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:58.526789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:58.526789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:58.526789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:58.526789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:58.526789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:58.526789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:58.526789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:58.526789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:58.526789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:58.526789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:58.526789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:58.527688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:58.527688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:58.527688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:58.527688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:58.527688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:15:58.527688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:15:58.527688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:15:58.527688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:15:58.527688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:15:58.527688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:15:58.527688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:15:58.527688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:15:58.528801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 263/543] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:15:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:15:58.534735 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fdcf8-da18-4cf0-81ba-8d9e91a247a7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:58.592651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4880286-5f36-4707-a6e7-f496c1601612 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:58.628246 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:58.660414 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4345607-716e-44fa-bef7-88522b65773c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:58.661590 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4345607-716e-44fa-bef7-88522b65773c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:58.662874 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4345607-716e-44fa-bef7-88522b65773c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:58.664015 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4345607-716e-44fa-bef7-88522b65773c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:58.665263 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4345607-716e-44fa-bef7-88522b65773c None admin] RBAC: Adding query filter params (role.id=f57b3e871ce142a090d6aa7dea9ac7d5, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=10d29b7934374f34923587eda9be0ed0) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:58.666933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4345607-716e-44fa-bef7-88522b65773c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:58.669239 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4345607-716e-44fa-bef7-88522b65773c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:58.676108 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 281/544] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:15:58 2022] GET /identity/v3/role_assignments?user.id=10d29b7934374f34923587eda9be0ed0&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=f57b3e871ce142a090d6aa7dea9ac7d5 => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:15:58.684536 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fdcf8-da18-4cf0-81ba-8d9e91a247a7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:58.718522 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283c1b0-331f-4019-bb7e-3b288a20212a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:58.719197 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283c1b0-331f-4019-bb7e-3b288a20212a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:58.719480 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283c1b0-331f-4019-bb7e-3b288a20212a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:58.719754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283c1b0-331f-4019-bb7e-3b288a20212a None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:58.720546 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283c1b0-331f-4019-bb7e-3b288a20212a None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:58.720837 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283c1b0-331f-4019-bb7e-3b288a20212a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:58.722242 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283c1b0-331f-4019-bb7e-3b288a20212a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:58.728067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:58.737628 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4345607-716e-44fa-bef7-88522b65773c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:58.744529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 264/545] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:15:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:58.859445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4345607-716e-44fa-bef7-88522b65773c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:58.911286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae2784-0422-48ed-a1f6-e8cbf4d65da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:58.912431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae2784-0422-48ed-a1f6-e8cbf4d65da7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:58.912944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae2784-0422-48ed-a1f6-e8cbf4d65da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:58.913953 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae2784-0422-48ed-a1f6-e8cbf4d65da7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:58.932389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae2784-0422-48ed-a1f6-e8cbf4d65da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:58.934200 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae2784-0422-48ed-a1f6-e8cbf4d65da7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:58.962631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ddae2784-0422-48ed-a1f6-e8cbf4d65da7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:58.962631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:58.962631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:58.962631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:58.962631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:58.962631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:58.962631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:58.962631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:58.962631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:58.962631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:58.962631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:58.962631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:58.962631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:15:58.962631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:15:58.962631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:15:58.963525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:15:58.963525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:58.963525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:58.963525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:58.963525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:58.963525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:58.963525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:58.963525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:58.963525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:58.963525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:58.963525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:58.963525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:58.963525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:58.963525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:58.963525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:58.963525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:58.963525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:58.964365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:15:58.964365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:15:58.964365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:15:58.964365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:58.966639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 282/546] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:58.972079 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283c1b0-331f-4019-bb7e-3b288a20212a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:59.115996 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283c1b0-331f-4019-bb7e-3b288a20212a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:59.171489 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd8cdb-efbe-4c98-a103-c5f62d64c420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:59.172098 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd8cdb-efbe-4c98-a103-c5f62d64c420 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:59.172387 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd8cdb-efbe-4c98-a103-c5f62d64c420 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:59.172697 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd8cdb-efbe-4c98-a103-c5f62d64c420 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:59.173433 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd8cdb-efbe-4c98-a103-c5f62d64c420 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:59.173743 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd8cdb-efbe-4c98-a103-c5f62d64c420 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:59.175116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd8cdb-efbe-4c98-a103-c5f62d64c420 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:59.194077 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 265/547] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:59.199242 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae2784-0422-48ed-a1f6-e8cbf4d65da7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:59.244061 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:59.321505 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae2784-0422-48ed-a1f6-e8cbf4d65da7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:59.376055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d29805-324c-4112-ac66-ffb4a4fcbb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:59.376902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d29805-324c-4112-ac66-ffb4a4fcbb5d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:59.377191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d29805-324c-4112-ac66-ffb4a4fcbb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:59.377470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d29805-324c-4112-ac66-ffb4a4fcbb5d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:59.385460 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d29805-324c-4112-ac66-ffb4a4fcbb5d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:59.386933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d29805-324c-4112-ac66-ffb4a4fcbb5d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:59.406429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-07d29805-324c-4112-ac66-ffb4a4fcbb5d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:15:59.406429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:15:59.406429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:15:59.406429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:15:59.406429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:15:59.406429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:15:59.406429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:15:59.406429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:15:59.406429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:15:59.406429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:15:59.406429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:15:59.406429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:15:59.406429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:15:59.406429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:15:59.406429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:15:59.406429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:15:59.407448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:15:59.407448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:15:59.407448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:15:59.407448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:15:59.407448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:15:59.407448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:15:59.407448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:15:59.407448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:15:59.407448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:15:59.407448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:15:59.407448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:15:59.407448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:15:59.407448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:15:59.407448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:15:59.407448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:15:59.407448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:15:59.407448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:15:59.407448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:15:59.409378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:15:59.409378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:15:59.409378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:15:59.409378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:15:59.409378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:15:59.409378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:15:59.409378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:15:59.409378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:15:59.409378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:15:59.409378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:15:59.409378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 283/548] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:15:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:15:59.413263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd8cdb-efbe-4c98-a103-c5f62d64c420 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:59.495906 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:59.551185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd8cdb-efbe-4c98-a103-c5f62d64c420 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:59.598803 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b88b1d-ba4b-41f6-9d18-daa70af998b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:59.599819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b88b1d-ba4b-41f6-9d18-daa70af998b7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:59.600301 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b88b1d-ba4b-41f6-9d18-daa70af998b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:59.600831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b88b1d-ba4b-41f6-9d18-daa70af998b7 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:59.601889 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b88b1d-ba4b-41f6-9d18-daa70af998b7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:15:59.602396 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b88b1d-ba4b-41f6-9d18-daa70af998b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:59.604017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b88b1d-ba4b-41f6-9d18-daa70af998b7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:59.624763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 266/549] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Jul 18 10:15:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:15:59.629875 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d29805-324c-4112-ac66-ffb4a4fcbb5d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:59.750310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48554ab8-7bfa-46c6-b2f1-9731a9e6ddef None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:59.750916 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48554ab8-7bfa-46c6-b2f1-9731a9e6ddef None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:59.751391 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48554ab8-7bfa-46c6-b2f1-9731a9e6ddef None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:59.754308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 267/550] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:15:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:15:59.759116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fe8d1-6074-492e-85eb-2d6fc6f1272f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:59.759652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fe8d1-6074-492e-85eb-2d6fc6f1272f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:59.760147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fe8d1-6074-492e-85eb-2d6fc6f1272f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:59.771889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:15:59.783179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-188fe8d1-6074-492e-85eb-2d6fc6f1272f None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:15:59.784277 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d29805-324c-4112-ac66-ffb4a4fcbb5d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:15:59.784995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-188fe8d1-6074-492e-85eb-2d6fc6f1272f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:15:59.861034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f6119-6c55-4bdd-9a35-83148cbba882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:15:59.861688 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f6119-6c55-4bdd-9a35-83148cbba882 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:15:59.861970 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f6119-6c55-4bdd-9a35-83148cbba882 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:15:59.862249 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f6119-6c55-4bdd-9a35-83148cbba882 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:15:59.883132 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 268/551] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:15:59.923170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fe8d1-6074-492e-85eb-2d6fc6f1272f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:15:59.968062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f6119-6c55-4bdd-9a35-83148cbba882 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:15:59.969577 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f6119-6c55-4bdd-9a35-83148cbba882 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:15:59.988903 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:00.041000 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fe8d1-6074-492e-85eb-2d6fc6f1272f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:00.104186 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed078c-2a01-487e-9107-47857d72702a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:00.105284 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed078c-2a01-487e-9107-47857d72702a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:00.105570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed078c-2a01-487e-9107-47857d72702a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:00.105849 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed078c-2a01-487e-9107-47857d72702a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:00.132096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed078c-2a01-487e-9107-47857d72702a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:00.150974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d5ed078c-2a01-487e-9107-47857d72702a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:00.150974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:00.150974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:00.150974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:00.150974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:00.150974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:00.150974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:00.150974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:00.150974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:00.150974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:00.150974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:00.150974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:00.150974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:00.150974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:00.150974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:00.150974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:00.152061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:00.152061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:00.152061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:00.152061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:00.152061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:00.152061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:00.152061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:00.152061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:00.152061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:00.152061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:00.152061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:00.152061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:00.152061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:00.152061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:00.152061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:00.152061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:00.152934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:00.152934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:00.152934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:00.152934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:00.152934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:00.152934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:00.152934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:00.152934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:00.152934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:00.152934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:00.152934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:00.152934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:00.153558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:00.164988 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 269/552] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:15:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:00.164988 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed078c-2a01-487e-9107-47857d72702a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:00.173620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 284/553] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:15:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:16:00.180173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f6119-6c55-4bdd-9a35-83148cbba882 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:00.290741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed078c-2a01-487e-9107-47857d72702a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:00.302297 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:00.336641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b2fdf-41c8-438f-9878-f4d548defdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:00.337525 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b2fdf-41c8-438f-9878-f4d548defdfa None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:00.338206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b2fdf-41c8-438f-9878-f4d548defdfa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:00.338962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b2fdf-41c8-438f-9878-f4d548defdfa None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:00.339840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b2fdf-41c8-438f-9878-f4d548defdfa None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:00.340249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b2fdf-41c8-438f-9878-f4d548defdfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:00.342304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b2fdf-41c8-438f-9878-f4d548defdfa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:00.353400 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 270/554] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:16:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:00.354019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f6119-6c55-4bdd-9a35-83148cbba882 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:00.357017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbabacc-b36d-40a6-a889-37ed4a4a86d0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:00.357310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbabacc-b36d-40a6-a889-37ed4a4a86d0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:00.357591 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbabacc-b36d-40a6-a889-37ed4a4a86d0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:00.358912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 271/555] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:00.359855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbabacc-b36d-40a6-a889-37ed4a4a86d0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:00.397193 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c61d01-c202-4036-b9ae-574f7174929b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:00.398438 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c61d01-c202-4036-b9ae-574f7174929b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:00.399074 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c61d01-c202-4036-b9ae-574f7174929b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:00.399568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c61d01-c202-4036-b9ae-574f7174929b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:00.410655 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c61d01-c202-4036-b9ae-574f7174929b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:00.412371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c61d01-c202-4036-b9ae-574f7174929b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:00.433440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a6c61d01-c202-4036-b9ae-574f7174929b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:00.433440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:00.433440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:00.433440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:00.433440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:00.433440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:00.433440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:00.433440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:00.433440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:00.433440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:00.433440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:00.433440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:00.433440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:00.433440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:00.433440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:00.434537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:00.434537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:00.434537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:00.434537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:00.434537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:00.434537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:00.434537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:00.434537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:00.434537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:00.434537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:00.434537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:00.434537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:00.434537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:00.434537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:00.434537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:00.434537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:00.434537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:00.435408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:00.435408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:00.435408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:00.435408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:00.435408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 285/556] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:00.436159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527f564-965a-4a56-b755-8afb10a94ee3 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:00.436260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527f564-965a-4a56-b755-8afb10a94ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:00.436537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7527f564-965a-4a56-b755-8afb10a94ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:00.453949 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7527f564-965a-4a56-b755-8afb10a94ee3 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:00.455144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7527f564-965a-4a56-b755-8afb10a94ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:00.489011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbabacc-b36d-40a6-a889-37ed4a4a86d0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:00.499327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:00.527135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d313a-01a5-4a9e-8072-5fbad81e8cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:00.527565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 286/557] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:16:00.527967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d313a-01a5-4a9e-8072-5fbad81e8cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:00.528257 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d313a-01a5-4a9e-8072-5fbad81e8cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:00.528560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d313a-01a5-4a9e-8072-5fbad81e8cb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:00.529756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d7c8f-5506-484e-b44c-d6f2dd39fc4d None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:00.530259 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d7c8f-5506-484e-b44c-d6f2dd39fc4d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:00.530731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d7c8f-5506-484e-b44c-d6f2dd39fc4d None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:00.532469 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 287/558] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:00.533379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d7c8f-5506-484e-b44c-d6f2dd39fc4d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:00.548910 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d313a-01a5-4a9e-8072-5fbad81e8cb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:00.551636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d313a-01a5-4a9e-8072-5fbad81e8cb2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:00.576194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-6d3d313a-01a5-4a9e-8072-5fbad81e8cb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:00.576194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:00.576194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:00.576194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:00.576194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:00.576194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:00.576194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:00.576194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:00.576194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:00.576194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:00.576194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:00.576194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:00.576194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:00.576194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:00.576194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:00.577818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:00.577818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:00.577818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:00.577818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:00.577818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:00.577818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:00.577818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:00.577818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:00.577818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:00.577818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:00.577818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:00.577818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:00.577818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:00.577818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:00.577818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:00.577818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:00.577818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:00.578709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:00.578709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:00.578709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:00.578709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:00.579363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 272/559] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:00.580807 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7a918-b161-4932-bf8f-7affdb549eb6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:00.581100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7a918-b161-4932-bf8f-7affdb549eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:00.581377 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7a918-b161-4932-bf8f-7affdb549eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:00.598398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9ed7a918-b161-4932-bf8f-7affdb549eb6 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:00.599592 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9ed7a918-b161-4932-bf8f-7affdb549eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:00.619118 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:00.661672 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d7c8f-5506-484e-b44c-d6f2dd39fc4d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:00.680530 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 273/560] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:16:00.680530 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7a918-b161-4932-bf8f-7affdb549eb6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:00.707027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0e286-061c-4a51-9bbf-c76d5069eb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:00.707749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0e286-061c-4a51-9bbf-c76d5069eb5e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:00.708031 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0e286-061c-4a51-9bbf-c76d5069eb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:00.708305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0e286-061c-4a51-9bbf-c76d5069eb5e None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:00.709067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa0e286-061c-4a51-9bbf-c76d5069eb5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:00.709373 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa0e286-061c-4a51-9bbf-c76d5069eb5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:00.710798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa0e286-061c-4a51-9bbf-c76d5069eb5e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:00.723264 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 288/561] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:00.728357 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0e286-061c-4a51-9bbf-c76d5069eb5e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:00.761025 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:00.786806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7a918-b161-4932-bf8f-7affdb549eb6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:00.823764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d6cb6-c79f-487a-998f-575ef624d76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:00.824680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d6cb6-c79f-487a-998f-575ef624d76a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:00.825151 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d6cb6-c79f-487a-998f-575ef624d76a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:00.825261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d6cb6-c79f-487a-998f-575ef624d76a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:00.833721 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0e286-061c-4a51-9bbf-c76d5069eb5e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:00.836317 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620d6cb6-c79f-487a-998f-575ef624d76a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:00.843902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-620d6cb6-c79f-487a-998f-575ef624d76a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:00.843902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:00.843902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:00.843902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:00.843902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:00.843902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:00.843902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:00.843902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:00.843902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:00.843902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:00.843902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:00.843902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:00.843902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:00.843902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:00.843902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:00.843902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:00.845085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:00.845085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:00.845085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:00.845085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:00.845085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:00.845085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:00.845085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:00.845085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:00.845085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:00.845085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:00.845085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:00.845085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:00.845085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:00.845085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:00.845085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:00.845085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:00.845989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:00.845989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:00.845989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:00.845989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:00.845989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:00.845989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:00.845989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:00.845989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:00.845989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:00.845989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:00.845989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:00.845989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:00.845989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 274/562] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:16:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:00.845989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d6cb6-c79f-487a-998f-575ef624d76a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:00.863950 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:00.874292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce161a2-93b0-4191-9b44-190fc86bfe49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:00.874954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce161a2-93b0-4191-9b44-190fc86bfe49 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:00.875245 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce161a2-93b0-4191-9b44-190fc86bfe49 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:00.875524 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce161a2-93b0-4191-9b44-190fc86bfe49 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:00.876270 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce161a2-93b0-4191-9b44-190fc86bfe49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:00.876639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce161a2-93b0-4191-9b44-190fc86bfe49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:00.878018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce161a2-93b0-4191-9b44-190fc86bfe49 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:00.890482 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 289/563] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:00.893484 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce161a2-93b0-4191-9b44-190fc86bfe49 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:00.956637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d6cb6-c79f-487a-998f-575ef624d76a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:00.990302 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c87229-2944-40ad-9a2a-393b6df31f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:00.991094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c87229-2944-40ad-9a2a-393b6df31f64 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:00.991886 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c87229-2944-40ad-9a2a-393b6df31f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:00.992166 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c87229-2944-40ad-9a2a-393b6df31f64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:00.994619 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:01.002461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce161a2-93b0-4191-9b44-190fc86bfe49 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:01.002881 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c87229-2944-40ad-9a2a-393b6df31f64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:01.011102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-68c87229-2944-40ad-9a2a-393b6df31f64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:01.011102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:01.011102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:01.011102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:01.011102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:01.011102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:01.011102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:01.011102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:01.011102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:01.011102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:01.011102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:01.011102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:01.011102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:01.011102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:01.011102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:01.011102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:01.012282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:01.012282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:01.012282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:01.012282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:01.012282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:01.012282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:01.012282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:01.012282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:01.012282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:01.012282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:01.012282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:01.012282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:01.012282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:01.012282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:01.012282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:01.012282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:01.013398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:01.013398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:01.013398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:01.013398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:01.013398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:01.013398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:01.013398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:01.013398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:01.013398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:01.013398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:01.013398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:01.013398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:01.017114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 275/564] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:16:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:01.017940 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c87229-2944-40ad-9a2a-393b6df31f64 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:01.039128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a83639-98b3-41e2-83e6-01342cff4b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:01.039995 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a83639-98b3-41e2-83e6-01342cff4b0d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:01.040347 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a83639-98b3-41e2-83e6-01342cff4b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:01.040706 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a83639-98b3-41e2-83e6-01342cff4b0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:01.047954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a83639-98b3-41e2-83e6-01342cff4b0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:01.049531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a83639-98b3-41e2-83e6-01342cff4b0d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:01.064138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-91a83639-98b3-41e2-83e6-01342cff4b0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:01.064138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:01.064138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:01.064138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:01.064138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:01.064138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:01.064138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:01.064138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:01.064138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:01.064138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:01.064138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:01.064138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:01.064138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:01.064138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:01.064138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:01.065307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:01.065307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:01.065307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:01.065307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:01.065307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:01.065307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:01.065307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:01.065307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:01.065307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:01.065307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:01.065307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:01.065307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:01.065307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:01.065307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:01.065307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:01.065307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:01.065307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:01.066470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:01.066470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:01.066470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:01.066470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:01.066470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:01.066470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:01.066470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:01.066470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:01.067161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 290/565] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:01.072833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a83639-98b3-41e2-83e6-01342cff4b0d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:01.097079 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:01.127449 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c87229-2944-40ad-9a2a-393b6df31f64 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:01.160645 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0ecdc-6e39-482e-98e1-f78e0e8a237d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:01.161314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0ecdc-6e39-482e-98e1-f78e0e8a237d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:01.161595 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0ecdc-6e39-482e-98e1-f78e0e8a237d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:01.161870 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0ecdc-6e39-482e-98e1-f78e0e8a237d None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:01.162614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0ecdc-6e39-482e-98e1-f78e0e8a237d None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:01.162917 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0ecdc-6e39-482e-98e1-f78e0e8a237d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:01.164904 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0ecdc-6e39-482e-98e1-f78e0e8a237d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:01.172915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 276/566] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:16:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:01.173136 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a83639-98b3-41e2-83e6-01342cff4b0d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:01.173696 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0ecdc-6e39-482e-98e1-f78e0e8a237d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:01.207520 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc91db-0ebf-4ac6-a1da-07dd1017ab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:01.208313 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc91db-0ebf-4ac6-a1da-07dd1017ab92 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:01.208624 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc91db-0ebf-4ac6-a1da-07dd1017ab92 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:01.208900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc91db-0ebf-4ac6-a1da-07dd1017ab92 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:01.216176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc91db-0ebf-4ac6-a1da-07dd1017ab92 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:01.217652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc91db-0ebf-4ac6-a1da-07dd1017ab92 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:01.232409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a3fc91db-0ebf-4ac6-a1da-07dd1017ab92 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 10:16:01.232409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:01.232409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:01.232409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:01.232409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:01.232409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:01.232409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:01.232409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:01.232409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:01.232409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:01.232409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:01.232409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:01.232409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:01.232409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:01.232409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:01.232409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:01.233582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:01.233582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:01.233582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:01.233582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:01.233582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:01.233582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:01.233582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:01.233582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:01.233582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:01.233582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:01.233582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:01.233582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:01.233582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:01.233582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:01.233582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:01.233582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:01.233582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:01.233582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:01.234881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:01.234881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:01.234881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:01.234881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:01.234881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:01.234881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:01.234881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:01.234881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:01.234881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 10:16:01.234881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:01.234881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 291/567] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:16:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:01.234881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc91db-0ebf-4ac6-a1da-07dd1017ab92 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:01.253831 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:01.274632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0ecdc-6e39-482e-98e1-f78e0e8a237d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:01.308069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535de38a-4d02-4239-9b9b-659f1f8db965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:01.308765 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535de38a-4d02-4239-9b9b-659f1f8db965 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:01.309044 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535de38a-4d02-4239-9b9b-659f1f8db965 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:01.309320 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535de38a-4d02-4239-9b9b-659f1f8db965 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:01.310062 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535de38a-4d02-4239-9b9b-659f1f8db965 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:01.310368 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535de38a-4d02-4239-9b9b-659f1f8db965 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:01.311791 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535de38a-4d02-4239-9b9b-659f1f8db965 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:01.313181 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:01.320464 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 277/568] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:16:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:01.321720 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c687b0d-8acd-45b8-b874-ca0bc5e16cdc None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:01.322008 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c687b0d-8acd-45b8-b874-ca0bc5e16cdc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:01.322284 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c687b0d-8acd-45b8-b874-ca0bc5e16cdc None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:01.327799 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 278/569] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:01.327799 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c687b0d-8acd-45b8-b874-ca0bc5e16cdc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:01.339623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc91db-0ebf-4ac6-a1da-07dd1017ab92 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:01.375761 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92002f-7cfe-440a-b081-81b08a7e0567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:01.376422 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92002f-7cfe-440a-b081-81b08a7e0567 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:01.376730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92002f-7cfe-440a-b081-81b08a7e0567 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:01.377005 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92002f-7cfe-440a-b081-81b08a7e0567 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:01.377794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92002f-7cfe-440a-b081-81b08a7e0567 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:01.378106 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92002f-7cfe-440a-b081-81b08a7e0567 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:01.379535 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92002f-7cfe-440a-b081-81b08a7e0567 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:01.391295 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 292/570] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:01.392072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92002f-7cfe-440a-b081-81b08a7e0567 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:01.424746 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c687b0d-8acd-45b8-b874-ca0bc5e16cdc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:01.457600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd5f84-44dc-40c0-9e7e-09c9168aa39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:01.458384 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd5f84-44dc-40c0-9e7e-09c9168aa39a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:01.458668 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd5f84-44dc-40c0-9e7e-09c9168aa39a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:01.458938 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd5f84-44dc-40c0-9e7e-09c9168aa39a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:01.464376 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:01.469055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd5f84-44dc-40c0-9e7e-09c9168aa39a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:01.470502 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd5f84-44dc-40c0-9e7e-09c9168aa39a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:01.488211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d5bd5f84-44dc-40c0-9e7e-09c9168aa39a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:01.488211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:01.488211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:01.488211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:01.488211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:01.488211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:01.488211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:01.488211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:01.488211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:01.488211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:01.488211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:01.488211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:01.488211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:01.488211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:01.488211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:01.489704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:01.489704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:01.489704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:01.489704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:01.489704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:01.489704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:01.489704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:01.489704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:01.489704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:01.489704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:01.489704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:01.489704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:01.489704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:01.489704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:01.489704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:01.489704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:01.489704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:01.490637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:01.490637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:01.490637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:01.490637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:01.490637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 279/571] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:01.490637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd5f84-44dc-40c0-9e7e-09c9168aa39a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:01.490975 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92002f-7cfe-440a-b081-81b08a7e0567 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:01.528185 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c157d-eadb-49c4-a5a1-c1c3a7b938f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:01.528881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c157d-eadb-49c4-a5a1-c1c3a7b938f8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:01.529170 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c157d-eadb-49c4-a5a1-c1c3a7b938f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:01.529443 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c157d-eadb-49c4-a5a1-c1c3a7b938f8 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:01.530288 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c157d-eadb-49c4-a5a1-c1c3a7b938f8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:01.530599 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c157d-eadb-49c4-a5a1-c1c3a7b938f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:01.532027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c157d-eadb-49c4-a5a1-c1c3a7b938f8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:01.538853 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:01.549239 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 293/572] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Jul 18 10:16:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:01.550437 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355ba8e-7025-4bd5-876e-7acd802f15c1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:01.550722 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355ba8e-7025-4bd5-876e-7acd802f15c1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:01.550996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355ba8e-7025-4bd5-876e-7acd802f15c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:01.566397 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2355ba8e-7025-4bd5-876e-7acd802f15c1 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:01.567576 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2355ba8e-7025-4bd5-876e-7acd802f15c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:01.589965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd5f84-44dc-40c0-9e7e-09c9168aa39a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:01.623209 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440cc43-1c59-42f1-a189-63714bc1eddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:01.624001 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440cc43-1c59-42f1-a189-63714bc1eddb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:01.624286 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440cc43-1c59-42f1-a189-63714bc1eddb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:01.624595 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440cc43-1c59-42f1-a189-63714bc1eddb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:01.634501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 294/573] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:16:01.635278 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355ba8e-7025-4bd5-876e-7acd802f15c1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:01.641773 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440cc43-1c59-42f1-a189-63714bc1eddb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:01.643423 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440cc43-1c59-42f1-a189-63714bc1eddb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:01.661693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d440cc43-1c59-42f1-a189-63714bc1eddb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:01.661693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:01.661693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:01.661693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:01.661693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:01.661693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:01.661693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:01.661693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:01.661693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:01.661693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:01.661693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:01.661693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:01.661693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:01.661693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:01.661693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:01.662795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:01.662795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:01.662795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:01.662795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:01.662795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:01.662795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:01.662795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:01.662795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:01.662795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:01.662795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:01.662795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:01.662795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:01.662795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:01.662795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:01.662795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:01.662795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:01.662795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:01.664446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:01.664446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:01.664446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:01.664446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:01.664446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 280/574] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:01.664446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440cc43-1c59-42f1-a189-63714bc1eddb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:01.697481 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:01.735490 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355ba8e-7025-4bd5-876e-7acd802f15c1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:01.760142 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:01.763187 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440cc43-1c59-42f1-a189-63714bc1eddb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:01.769333 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16813ec0-4d8c-40ce-a927-332e0198f656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:01.770050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16813ec0-4d8c-40ce-a927-332e0198f656 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:01.770405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16813ec0-4d8c-40ce-a927-332e0198f656 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:01.770765 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16813ec0-4d8c-40ce-a927-332e0198f656 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:01.771724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16813ec0-4d8c-40ce-a927-332e0198f656 None admin] RBAC: Adding query filter params (role.id=f57b3e871ce142a090d6aa7dea9ac7d5, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=fd6168873f2846c8af73b903bba17eb8) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:01.772041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16813ec0-4d8c-40ce-a927-332e0198f656 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:01.773706 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16813ec0-4d8c-40ce-a927-332e0198f656 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:01.779319 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 295/575] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:16:01 2022] GET /identity/v3/role_assignments?user.id=fd6168873f2846c8af73b903bba17eb8&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=f57b3e871ce142a090d6aa7dea9ac7d5 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:01.779906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16813ec0-4d8c-40ce-a927-332e0198f656 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:01.809993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e721c-17ed-4b7a-86da-195d607b1a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:01.810677 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e721c-17ed-4b7a-86da-195d607b1a11 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:01.810968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e721c-17ed-4b7a-86da-195d607b1a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:01.811250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e721c-17ed-4b7a-86da-195d607b1a11 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:01.812000 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e721c-17ed-4b7a-86da-195d607b1a11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:01.812315 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e721c-17ed-4b7a-86da-195d607b1a11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:01.813787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e721c-17ed-4b7a-86da-195d607b1a11 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:01.827557 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 281/576] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:01.828449 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e721c-17ed-4b7a-86da-195d607b1a11 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:01.897363 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16813ec0-4d8c-40ce-a927-332e0198f656 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:01.925007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:01.937589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c3b24-c7c2-4c48-88fe-24f50b1a02e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:01.938616 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c3b24-c7c2-4c48-88fe-24f50b1a02e0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:01.938901 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c3b24-c7c2-4c48-88fe-24f50b1a02e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:01.939806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c3b24-c7c2-4c48-88fe-24f50b1a02e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:01.940224 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e721c-17ed-4b7a-86da-195d607b1a11 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:01.954450 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c3b24-c7c2-4c48-88fe-24f50b1a02e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:01.956054 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c3b24-c7c2-4c48-88fe-24f50b1a02e0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:01.974471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-af3c3b24-c7c2-4c48-88fe-24f50b1a02e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:01.974471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:01.974471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:01.974471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:01.974471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:01.974471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:01.974471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:01.974471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:01.974471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:01.974471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:01.974471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:01.974471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:01.974471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:01.974471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:01.974471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:01.975620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:01.975620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:01.975620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:01.975620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:01.975620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:01.975620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:01.975620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:01.975620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:01.975620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:01.975620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:01.975620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:01.975620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:01.975620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:01.975620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:01.975620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:01.975620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:01.975620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:01.976777 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:01.976777 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:01.976777 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:01.976777 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:01.978294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 296/577] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:01.979075 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c3b24-c7c2-4c48-88fe-24f50b1a02e0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:01.989711 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704e463-d137-4565-b5d1-951bfaff021b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:01.990617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704e463-d137-4565-b5d1-951bfaff021b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:01.990978 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704e463-d137-4565-b5d1-951bfaff021b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:01.991462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704e463-d137-4565-b5d1-951bfaff021b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:02.004350 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704e463-d137-4565-b5d1-951bfaff021b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:02.012617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0704e463-d137-4565-b5d1-951bfaff021b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:02.012617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:02.012617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:02.012617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:02.012617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:02.012617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:02.012617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:02.012617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:02.012617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:02.012617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:02.012617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:02.012617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:02.012617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:02.012617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:02.012617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:02.012617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:02.013545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:02.013545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:02.013545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:02.013545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:02.013545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:02.013545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:02.013545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:02.013545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:02.013545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:02.013545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:02.013545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:02.013545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:02.013545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:02.013545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:02.013545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:02.013545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:02.014389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:02.014389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:02.014389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:02.014389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:02.014389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:02.014389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:02.014389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:02.014389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:02.014389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:02.014389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:02.014389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:02.014389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:02.014389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 282/578] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:16:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:02.015069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704e463-d137-4565-b5d1-951bfaff021b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:02.039689 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:02.093050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c3b24-c7c2-4c48-88fe-24f50b1a02e0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:02.122940 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704e463-d137-4565-b5d1-951bfaff021b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:02.129494 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce688dd-118b-44da-bab3-8c1066240fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:02.130283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce688dd-118b-44da-bab3-8c1066240fcf None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:02.130682 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce688dd-118b-44da-bab3-8c1066240fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:02.131056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce688dd-118b-44da-bab3-8c1066240fcf None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:02.132409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce688dd-118b-44da-bab3-8c1066240fcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:02.132846 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce688dd-118b-44da-bab3-8c1066240fcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:02.134335 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce688dd-118b-44da-bab3-8c1066240fcf None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:02.146026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 297/579] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:02.146924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce688dd-118b-44da-bab3-8c1066240fcf None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:02.165223 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43807da4-6b87-46c5-beae-1fdf11bf299f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:02.166022 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43807da4-6b87-46c5-beae-1fdf11bf299f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:02.166304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43807da4-6b87-46c5-beae-1fdf11bf299f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:02.166572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43807da4-6b87-46c5-beae-1fdf11bf299f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:02.177411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43807da4-6b87-46c5-beae-1fdf11bf299f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:02.177411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43807da4-6b87-46c5-beae-1fdf11bf299f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:02.192538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-43807da4-6b87-46c5-beae-1fdf11bf299f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:16:02.192538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:02.192538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:02.192538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:02.192538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:02.192538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:02.192538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:02.192538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:02.192538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:02.192538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:02.192538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:02.192538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:02.192538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:02.192538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:02.192538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:02.192538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:02.193506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:02.193506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:02.193506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:02.193506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:02.193506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:02.193506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:02.193506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:02.193506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:02.193506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:02.193506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:02.193506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:02.193506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:02.193506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:02.193506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:02.193506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:02.193506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:02.193506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:02.193506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:02.195029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:02.195029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:02.195029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:02.195029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:02.195029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:02.195029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:02.195029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:02.195029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:02.195029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:16:02.195029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:02.198200 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:02.198863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 283/580] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:16:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:02.199742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43807da4-6b87-46c5-beae-1fdf11bf299f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:02.263770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce688dd-118b-44da-bab3-8c1066240fcf None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:02.289085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:02.300607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25527cd2-9841-430b-919c-b61f400ffda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:02.301270 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25527cd2-9841-430b-919c-b61f400ffda3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:02.301555 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25527cd2-9841-430b-919c-b61f400ffda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:02.302062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25527cd2-9841-430b-919c-b61f400ffda3 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:02.302548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25527cd2-9841-430b-919c-b61f400ffda3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:02.302853 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25527cd2-9841-430b-919c-b61f400ffda3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:02.304761 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25527cd2-9841-430b-919c-b61f400ffda3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:02.311370 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43807da4-6b87-46c5-beae-1fdf11bf299f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:02.316255 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 298/581] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:02.317284 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25527cd2-9841-430b-919c-b61f400ffda3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:02.349042 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e8ecc-8712-4499-8342-07fbb8c1080f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:02.349701 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e8ecc-8712-4499-8342-07fbb8c1080f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:02.349987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e8ecc-8712-4499-8342-07fbb8c1080f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:02.350275 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e8ecc-8712-4499-8342-07fbb8c1080f None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:02.351179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e8ecc-8712-4499-8342-07fbb8c1080f None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:02.351339 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e8ecc-8712-4499-8342-07fbb8c1080f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:02.352736 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e8ecc-8712-4499-8342-07fbb8c1080f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:02.365278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 284/582] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:16:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:02.365278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e8ecc-8712-4499-8342-07fbb8c1080f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:02.425176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:02.429691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25527cd2-9841-430b-919c-b61f400ffda3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:02.473645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0500e3-0b2e-4e1e-9ec2-fc1b918fb885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:02.474623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0500e3-0b2e-4e1e-9ec2-fc1b918fb885 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:02.475002 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0500e3-0b2e-4e1e-9ec2-fc1b918fb885 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:02.475380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0500e3-0b2e-4e1e-9ec2-fc1b918fb885 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:02.481834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e8ecc-8712-4499-8342-07fbb8c1080f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:02.484061 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0500e3-0b2e-4e1e-9ec2-fc1b918fb885 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:02.485650 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0500e3-0b2e-4e1e-9ec2-fc1b918fb885 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:02.500119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-3d0500e3-0b2e-4e1e-9ec2-fc1b918fb885 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:16:02.500119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:02.500119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:02.500119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:02.500119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:02.500119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:02.500119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:02.500119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:02.500119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:02.500119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:02.500119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:02.500119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:02.500119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:02.500119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:02.500119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:02.500119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:02.501299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:02.501299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:02.501299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:02.501299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:02.501299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:02.501299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:02.501299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:02.501299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:02.501299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:02.501299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:02.501299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:02.501299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:02.501299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:02.501299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:02.501299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:02.501299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:02.501299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:02.501299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:02.502485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:02.502485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:02.502485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:02.502485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:02.502485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:02.502485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:02.502485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:02.502485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:02.502485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:16:02.502485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:02.508719 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 299/583] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jul 18 10:16:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:02.508719 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0500e3-0b2e-4e1e-9ec2-fc1b918fb885 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:02.523303 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e3919-8779-4156-98c9-556a7824fadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:02.524613 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e3919-8779-4156-98c9-556a7824fadb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:02.525047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e3919-8779-4156-98c9-556a7824fadb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:02.525317 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e3919-8779-4156-98c9-556a7824fadb None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:02.526160 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e3919-8779-4156-98c9-556a7824fadb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:02.526475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e3919-8779-4156-98c9-556a7824fadb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:02.527906 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e3919-8779-4156-98c9-556a7824fadb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:02.543693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:02.547667 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 285/584] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Jul 18 10:16:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:02.548412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e3919-8779-4156-98c9-556a7824fadb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:02.621931 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0500e3-0b2e-4e1e-9ec2-fc1b918fb885 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:02.655285 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e3919-8779-4156-98c9-556a7824fadb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:02.667506 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32719ad-02e0-4634-ab7f-ed86f751b0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:02.668304 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32719ad-02e0-4634-ab7f-ed86f751b0df None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:02.668619 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32719ad-02e0-4634-ab7f-ed86f751b0df None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:02.668896 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32719ad-02e0-4634-ab7f-ed86f751b0df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:02.675943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32719ad-02e0-4634-ab7f-ed86f751b0df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:02.677352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32719ad-02e0-4634-ab7f-ed86f751b0df None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:02.689409 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6809d-e099-44b9-a772-adc2152fbe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:02.690190 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6809d-e099-44b9-a772-adc2152fbe59 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:02.690472 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6809d-e099-44b9-a772-adc2152fbe59 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:02.690744 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6809d-e099-44b9-a772-adc2152fbe59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:02.691271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b32719ad-02e0-4634-ab7f-ed86f751b0df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:02.691271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:02.691271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:02.691271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:02.691271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:02.691271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:02.691271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:02.691271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:02.691271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:02.691271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:02.691271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:02.691271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:02.691271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:02.691271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:02.691271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:02.692117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:02.692117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:02.692117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:02.692117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:02.692117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:02.692117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:02.692117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:02.692117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:02.692117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:02.692117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:02.692117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:02.692117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:02.692117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:02.692117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:02.692117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:02.692117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:02.692117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:02.693107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:02.693107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:02.693107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:02.693107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:02.693107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:02.693107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:02.693107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:02.693107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:02.693107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 300/585] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:02.693594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32719ad-02e0-4634-ab7f-ed86f751b0df None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:02.707181 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6809d-e099-44b9-a772-adc2152fbe59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:02.708647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6809d-e099-44b9-a772-adc2152fbe59 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:02.713665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:02.727078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-60e6809d-e099-44b9-a772-adc2152fbe59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:02.727078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:02.727078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:02.727078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:02.727078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:02.727078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:02.727078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:02.727078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:02.727078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:02.727078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:02.727078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:02.727078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:02.727078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:02.727078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:02.727078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:02.727976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:02.727976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:02.727976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:02.727976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:02.727976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:02.727976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:02.727976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:02.727976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:02.727976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:02.727976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:02.727976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:02.727976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:02.727976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:02.727976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:02.727976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:02.727976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:02.727976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:02.728908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:02.728908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:02.728908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:02.728908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:02.728908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 286/586] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:02.730195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6809d-e099-44b9-a772-adc2152fbe59 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:02.784305 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:02.805452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32719ad-02e0-4634-ab7f-ed86f751b0df None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:02.833284 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6809d-e099-44b9-a772-adc2152fbe59 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:02.842984 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0b05b-8c2e-446c-bf94-25b101feb189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:02.843842 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0b05b-8c2e-446c-bf94-25b101feb189 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:02.844200 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0b05b-8c2e-446c-bf94-25b101feb189 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:02.844563 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0b05b-8c2e-446c-bf94-25b101feb189 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:02.845479 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b0b05b-8c2e-446c-bf94-25b101feb189 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:02.845857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b0b05b-8c2e-446c-bf94-25b101feb189 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:02.847594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b0b05b-8c2e-446c-bf94-25b101feb189 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:02.865932 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 301/587] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jul 18 10:16:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:02.866939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0b05b-8c2e-446c-bf94-25b101feb189 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:02.871973 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5daa4c0-029b-48b2-8684-7fb9fe90d227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:02.872895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5daa4c0-029b-48b2-8684-7fb9fe90d227 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:02.873248 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5daa4c0-029b-48b2-8684-7fb9fe90d227 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:02.873593 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5daa4c0-029b-48b2-8684-7fb9fe90d227 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:02.885978 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5daa4c0-029b-48b2-8684-7fb9fe90d227 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:02.887486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5daa4c0-029b-48b2-8684-7fb9fe90d227 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:02.905512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b5daa4c0-029b-48b2-8684-7fb9fe90d227 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:02.905512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:02.905512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:02.905512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:02.905512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:02.905512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:02.905512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:02.905512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:02.905512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:02.905512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:02.905512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:02.905512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:02.905512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:02.905512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:02.905512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:02.906436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:02.906436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:02.906436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:02.906436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:02.906436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:02.906436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:02.906436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:02.906436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:02.906436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:02.906436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:02.906436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:02.906436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:02.906436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:02.906436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:02.906436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:02.906436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:02.906436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:02.907342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:02.907342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:02.907342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:02.907342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:02.907342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 287/588] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:02.907613 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5daa4c0-029b-48b2-8684-7fb9fe90d227 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:02.956678 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:02.982123 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0b05b-8c2e-446c-bf94-25b101feb189 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:03.018437 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3c3c0-97e8-4be4-a87e-b3a410284269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:03.019233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3c3c0-97e8-4be4-a87e-b3a410284269 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:03.019340 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5daa4c0-029b-48b2-8684-7fb9fe90d227 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:03.019688 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3c3c0-97e8-4be4-a87e-b3a410284269 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:03.020057 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3c3c0-97e8-4be4-a87e-b3a410284269 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:03.020970 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3c3c0-97e8-4be4-a87e-b3a410284269 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:03.021374 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3c3c0-97e8-4be4-a87e-b3a410284269 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:03.022490 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:03.023948 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3c3c0-97e8-4be4-a87e-b3a410284269 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:03.040680 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 302/589] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:03.040680 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3c3c0-97e8-4be4-a87e-b3a410284269 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:03.061866 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ccb67-32a3-4f1d-8ea7-cadea9bd1b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:03.062697 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ccb67-32a3-4f1d-8ea7-cadea9bd1b00 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:03.062992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ccb67-32a3-4f1d-8ea7-cadea9bd1b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:03.063375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ccb67-32a3-4f1d-8ea7-cadea9bd1b00 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:03.064277 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ccb67-32a3-4f1d-8ea7-cadea9bd1b00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:03.064754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ccb67-32a3-4f1d-8ea7-cadea9bd1b00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:03.066224 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ccb67-32a3-4f1d-8ea7-cadea9bd1b00 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:03.083698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 288/590] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:03.083698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ccb67-32a3-4f1d-8ea7-cadea9bd1b00 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:03.143161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3c3c0-97e8-4be4-a87e-b3a410284269 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:03.182662 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230151ee-e116-4819-8bce-2a0cfe95b1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:03.183457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230151ee-e116-4819-8bce-2a0cfe95b1cc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:03.183738 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230151ee-e116-4819-8bce-2a0cfe95b1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:03.184016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230151ee-e116-4819-8bce-2a0cfe95b1cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:03.186780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:03.191306 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ccb67-32a3-4f1d-8ea7-cadea9bd1b00 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:03.197279 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230151ee-e116-4819-8bce-2a0cfe95b1cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:03.198788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230151ee-e116-4819-8bce-2a0cfe95b1cc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:03.217298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-230151ee-e116-4819-8bce-2a0cfe95b1cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:03.217298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:03.217298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:03.217298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:03.217298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:03.217298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:03.217298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:03.217298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:03.217298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:03.217298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:03.217298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:03.217298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:03.217298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:03.217298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:03.217298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:03.218175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:03.218175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:03.218175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:03.218175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:03.218175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:03.218175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:03.218175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:03.218175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:03.218175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:03.218175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:03.218175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:03.218175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:03.218175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:03.218175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:03.218175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:03.218175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:03.218175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:03.223473 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:03.223473 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:03.223473 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:03.223473 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:03.223473 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 303/591] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:03.223473 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230151ee-e116-4819-8bce-2a0cfe95b1cc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:03.234695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ff963-d7d2-47b1-9de0-f20b38aaf4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:03.235444 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ff963-d7d2-47b1-9de0-f20b38aaf4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:03.235737 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ff963-d7d2-47b1-9de0-f20b38aaf4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:03.236009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ff963-d7d2-47b1-9de0-f20b38aaf4e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:03.236763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ff963-d7d2-47b1-9de0-f20b38aaf4e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:03.237073 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ff963-d7d2-47b1-9de0-f20b38aaf4e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:03.238439 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ff963-d7d2-47b1-9de0-f20b38aaf4e3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:03.251149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 289/592] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:03.252050 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ff963-d7d2-47b1-9de0-f20b38aaf4e3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:03.277997 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:03.330696 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230151ee-e116-4819-8bce-2a0cfe95b1cc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:03.363721 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32a96f-2d7c-4f3e-9251-a85614068369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:03.364648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32a96f-2d7c-4f3e-9251-a85614068369 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:03.364916 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ff963-d7d2-47b1-9de0-f20b38aaf4e3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:03.365025 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32a96f-2d7c-4f3e-9251-a85614068369 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:03.365189 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32a96f-2d7c-4f3e-9251-a85614068369 None admin] PATH_INFO: `/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/3cee4d23797e430e9eeddc394ed7f50e/roles/f57b3e871ce142a090d6aa7dea9ac7d5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:03.380156 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32a96f-2d7c-4f3e-9251-a85614068369 None admin] RBAC: Authorizing `identity:create_grant(project_id=e71a3acd0adf46a2a35b965a27c4998f, user_id=3cee4d23797e430e9eeddc394ed7f50e, role_id=f57b3e871ce142a090d6aa7dea9ac7d5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:03.381646 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32a96f-2d7c-4f3e-9251-a85614068369 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:03.395126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 304/593] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:16:03 2022] PUT /identity/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/3cee4d23797e430e9eeddc394ed7f50e/roles/f57b3e871ce142a090d6aa7dea9ac7d5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:16:03.395945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32a96f-2d7c-4f3e-9251-a85614068369 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:03.412629 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed285d0-a634-41dd-9ac6-2956dc2513cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:03.414528 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed285d0-a634-41dd-9ac6-2956dc2513cb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:03.414845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed285d0-a634-41dd-9ac6-2956dc2513cb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:03.415129 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed285d0-a634-41dd-9ac6-2956dc2513cb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:03.424103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed285d0-a634-41dd-9ac6-2956dc2513cb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:03.425779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed285d0-a634-41dd-9ac6-2956dc2513cb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:03.443188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-fed285d0-a634-41dd-9ac6-2956dc2513cb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 10:16:03.443188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:03.443188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:03.443188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:03.443188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:03.443188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:03.443188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:03.443188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:03.443188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:03.443188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:03.443188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:03.443188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:03.443188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:03.443188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:03.443188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:03.443188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:03.444218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:03.444218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:03.444218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:03.444218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:03.444218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:03.444218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:03.444218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:03.444218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:03.444218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:03.444218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:03.444218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:03.444218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:03.444218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:03.444218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:03.444218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:03.444218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:03.444218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:03.444218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:03.445262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:03.445262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:03.445262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:03.445262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:03.445262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:03.445262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:03.445262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:03.445262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:03.445262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 10:16:03.445262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:03.445262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 290/594] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:16:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:03.445884 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed285d0-a634-41dd-9ac6-2956dc2513cb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:03.453184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:03.526189 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32a96f-2d7c-4f3e-9251-a85614068369 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:03.543736 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:03.564012 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed285d0-a634-41dd-9ac6-2956dc2513cb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:03.579110 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f4ee6-3d1e-477b-aae3-1cf25a3ddc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:03.580529 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f4ee6-3d1e-477b-aae3-1cf25a3ddc9d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:03.581588 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f4ee6-3d1e-477b-aae3-1cf25a3ddc9d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:03.582203 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f4ee6-3d1e-477b-aae3-1cf25a3ddc9d None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:03.583306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f4ee6-3d1e-477b-aae3-1cf25a3ddc9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:03.584071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f4ee6-3d1e-477b-aae3-1cf25a3ddc9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:03.586377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f4ee6-3d1e-477b-aae3-1cf25a3ddc9d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:03.603525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 305/595] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:03.604935 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f4ee6-3d1e-477b-aae3-1cf25a3ddc9d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:03.609985 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e2a5e-8d7c-4f1b-8696-f02301053986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:03.610857 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e2a5e-8d7c-4f1b-8696-f02301053986 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:03.611991 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e2a5e-8d7c-4f1b-8696-f02301053986 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:03.611991 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e2a5e-8d7c-4f1b-8696-f02301053986 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:03.620046 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e2a5e-8d7c-4f1b-8696-f02301053986 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:03.621788 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e2a5e-8d7c-4f1b-8696-f02301053986 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:03.637167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c90e2a5e-8d7c-4f1b-8696-f02301053986 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:03.637167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:03.637167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:03.637167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:03.637167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:03.637167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:03.637167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:03.637167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:03.637167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:03.637167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:03.637167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:03.637167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:03.637167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:03.637167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:03.637167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:03.638096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:03.638096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:03.638096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:03.638096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:03.638096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:03.638096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:03.638096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:03.638096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:03.638096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:03.638096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:03.638096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:03.638096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:03.638096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:03.638096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:03.638096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:03.638096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:03.638096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:03.638983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:03.638983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:03.638983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:03.638983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:03.638983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:03.638983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:03.638983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:03.638983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:03.638983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 291/596] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:03.639465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e2a5e-8d7c-4f1b-8696-f02301053986 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:03.740106 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:03.746678 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f4ee6-3d1e-477b-aae3-1cf25a3ddc9d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:03.763511 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e2a5e-8d7c-4f1b-8696-f02301053986 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:03.793737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29538b-58a5-4a68-9899-8c175503d0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:03.794928 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29538b-58a5-4a68-9899-8c175503d0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:03.795677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29538b-58a5-4a68-9899-8c175503d0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:03.796537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29538b-58a5-4a68-9899-8c175503d0f4 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:03.797919 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29538b-58a5-4a68-9899-8c175503d0f4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:03.798417 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29538b-58a5-4a68-9899-8c175503d0f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:03.800908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29538b-58a5-4a68-9899-8c175503d0f4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:03.805320 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33360c-99cb-4582-9be4-cd2c1d4cc48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:03.806213 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33360c-99cb-4582-9be4-cd2c1d4cc48a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:03.806569 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33360c-99cb-4582-9be4-cd2c1d4cc48a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:03.806914 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33360c-99cb-4582-9be4-cd2c1d4cc48a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:03.815765 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33360c-99cb-4582-9be4-cd2c1d4cc48a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:03.817246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33360c-99cb-4582-9be4-cd2c1d4cc48a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:03.823081 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 306/597] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Jul 18 10:16:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:03.824440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29538b-58a5-4a68-9899-8c175503d0f4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:03.832155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-2b33360c-99cb-4582-9be4-cd2c1d4cc48a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:03.832155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:03.832155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:03.832155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:03.832155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:03.832155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:03.832155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:03.832155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:03.832155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:03.832155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:03.832155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:03.832155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:03.832155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:03.832155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:03.832155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:03.833185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:03.833185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:03.833185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:03.833185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:03.833185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:03.833185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:03.833185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:03.833185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:03.833185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:03.833185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:03.833185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:03.833185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:03.833185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:03.833185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:03.833185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:03.833185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:03.833185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:03.839100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:03.839100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:03.839100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:03.839100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:03.839100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:03.839100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:03.839100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:03.839100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:03.839100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 292/598] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:03.839100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97c2a5-f953-4ea8-ba76-16a5bd5c3ffa None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:03.839100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97c2a5-f953-4ea8-ba76-16a5bd5c3ffa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:03.839733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97c2a5-f953-4ea8-ba76-16a5bd5c3ffa None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:03.839733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 293/599] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:03.839733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97c2a5-f953-4ea8-ba76-16a5bd5c3ffa None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:03.843520 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:03.934261 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29538b-58a5-4a68-9899-8c175503d0f4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:03.950476 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97c2a5-f953-4ea8-ba76-16a5bd5c3ffa None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:03.969666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10a2e4-6297-4e31-b33f-779c2acc3044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:03.970586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10a2e4-6297-4e31-b33f-779c2acc3044 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:03.970877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10a2e4-6297-4e31-b33f-779c2acc3044 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:03.971150 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10a2e4-6297-4e31-b33f-779c2acc3044 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:03.971939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10a2e4-6297-4e31-b33f-779c2acc3044 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:03.972248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10a2e4-6297-4e31-b33f-779c2acc3044 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:03.973694 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10a2e4-6297-4e31-b33f-779c2acc3044 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:03.986548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 307/600] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:03.987610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10a2e4-6297-4e31-b33f-779c2acc3044 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:03.995366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1fc5b-f78d-402a-9c30-4601a6515a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:03.996165 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1fc5b-f78d-402a-9c30-4601a6515a98 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:03.996448 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1fc5b-f78d-402a-9c30-4601a6515a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:03.996754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1fc5b-f78d-402a-9c30-4601a6515a98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:04.006364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:04.008468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1fc5b-f78d-402a-9c30-4601a6515a98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:04.010048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1fc5b-f78d-402a-9c30-4601a6515a98 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:04.028274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-60e1fc5b-f78d-402a-9c30-4601a6515a98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:04.028274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:04.028274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:04.028274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:04.028274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:04.028274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:04.028274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:04.028274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:04.028274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:04.028274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:04.028274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:04.028274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:04.028274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:04.028274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:04.028274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:04.029225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:04.029225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:04.029225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:04.029225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:04.029225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:04.029225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:04.029225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:04.029225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:04.029225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:04.029225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:04.029225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:04.029225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:04.029225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:04.029225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:04.029225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:04.029225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:04.029225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:04.030115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:04.030115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:04.030115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:04.030115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:04.030115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 294/601] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:04.030827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765512e4-7c36-4d4e-8245-e5e7ef5bd078 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:04.030924 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765512e4-7c36-4d4e-8245-e5e7ef5bd078 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:04.031237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765512e4-7c36-4d4e-8245-e5e7ef5bd078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:04.048437 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-765512e4-7c36-4d4e-8245-e5e7ef5bd078 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:04.049570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-765512e4-7c36-4d4e-8245-e5e7ef5bd078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:04.101774 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:04.103286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10a2e4-6297-4e31-b33f-779c2acc3044 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:04.119871 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 295/602] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:16:04.121347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765512e4-7c36-4d4e-8245-e5e7ef5bd078 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:04.140982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01569eb1-83ec-42d6-a46e-a3af94561320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:04.141902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01569eb1-83ec-42d6-a46e-a3af94561320 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:04.142190 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01569eb1-83ec-42d6-a46e-a3af94561320 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:04.142465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01569eb1-83ec-42d6-a46e-a3af94561320 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:04.143248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01569eb1-83ec-42d6-a46e-a3af94561320 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:04.143556 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01569eb1-83ec-42d6-a46e-a3af94561320 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:04.145116 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01569eb1-83ec-42d6-a46e-a3af94561320 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:04.164877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 308/603] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:04.164877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01569eb1-83ec-42d6-a46e-a3af94561320 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:04.235805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765512e4-7c36-4d4e-8245-e5e7ef5bd078 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:04.252894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:04.272060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deafb332-8d53-45ef-a63e-384a4c79ea5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:04.273055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deafb332-8d53-45ef-a63e-384a4c79ea5f None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:04.273336 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deafb332-8d53-45ef-a63e-384a4c79ea5f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:04.273616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deafb332-8d53-45ef-a63e-384a4c79ea5f None admin] PATH_INFO: `/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/20e29c37dc924514a882f936f5936ae1/roles/f57b3e871ce142a090d6aa7dea9ac7d5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:04.277458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01569eb1-83ec-42d6-a46e-a3af94561320 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:04.287487 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deafb332-8d53-45ef-a63e-384a4c79ea5f None admin] RBAC: Authorizing `identity:create_grant(project_id=e71a3acd0adf46a2a35b965a27c4998f, user_id=20e29c37dc924514a882f936f5936ae1, role_id=f57b3e871ce142a090d6aa7dea9ac7d5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:04.289485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deafb332-8d53-45ef-a63e-384a4c79ea5f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:04.308627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 296/604] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:16:04 2022] PUT /identity/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/20e29c37dc924514a882f936f5936ae1/roles/f57b3e871ce142a090d6aa7dea9ac7d5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:16:04.308627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deafb332-8d53-45ef-a63e-384a4c79ea5f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:04.336142 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca3dd1-0143-4151-9a86-0c14ad752525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:04.337020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca3dd1-0143-4151-9a86-0c14ad752525 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:04.337372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca3dd1-0143-4151-9a86-0c14ad752525 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:04.337756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca3dd1-0143-4151-9a86-0c14ad752525 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:04.339377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca3dd1-0143-4151-9a86-0c14ad752525 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:04.339763 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca3dd1-0143-4151-9a86-0c14ad752525 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:04.341953 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca3dd1-0143-4151-9a86-0c14ad752525 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:04.355472 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:04.361149 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 309/605] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:04.362074 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca3dd1-0143-4151-9a86-0c14ad752525 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:04.454118 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deafb332-8d53-45ef-a63e-384a4c79ea5f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:04.495924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca3dd1-0143-4151-9a86-0c14ad752525 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:04.500813 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f899cbd-9b63-47a6-9d1f-8ee43b48c136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:04.501618 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f899cbd-9b63-47a6-9d1f-8ee43b48c136 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:04.501948 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f899cbd-9b63-47a6-9d1f-8ee43b48c136 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:04.502229 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f899cbd-9b63-47a6-9d1f-8ee43b48c136 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:04.515379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f899cbd-9b63-47a6-9d1f-8ee43b48c136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:04.525707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-6f899cbd-9b63-47a6-9d1f-8ee43b48c136 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:04.525707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:04.525707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:04.525707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:04.525707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:04.525707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:04.525707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:04.525707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:04.525707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:04.525707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:04.525707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:04.525707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:04.525707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:04.525707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:04.525707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:04.525707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:04.526658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:04.526658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:04.526658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:04.526658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:04.526658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:04.526658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:04.526658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:04.526658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:04.526658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:04.526658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:04.526658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:04.526658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:04.526658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:04.526658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:04.526658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:04.526658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:04.527521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:04.527521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:04.527521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:04.527521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:04.527521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:04.527521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:04.527521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:04.527521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:04.527521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:04.527521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:04.527521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:04.527521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:04.528173 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 297/606] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:16:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:04.528733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f899cbd-9b63-47a6-9d1f-8ee43b48c136 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:04.545107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35515a9f-083a-4cf1-aefb-fee26916c972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:04.546465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35515a9f-083a-4cf1-aefb-fee26916c972 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:04.546754 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35515a9f-083a-4cf1-aefb-fee26916c972 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:04.547445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35515a9f-083a-4cf1-aefb-fee26916c972 None admin] PATH_INFO: `/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/29048c2021e340b081ae44a390583e7a/roles/f57b3e871ce142a090d6aa7dea9ac7d5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:04.552173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:04.569948 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35515a9f-083a-4cf1-aefb-fee26916c972 None admin] RBAC: Authorizing `identity:create_grant(project_id=e71a3acd0adf46a2a35b965a27c4998f, user_id=29048c2021e340b081ae44a390583e7a, role_id=f57b3e871ce142a090d6aa7dea9ac7d5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:04.572457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35515a9f-083a-4cf1-aefb-fee26916c972 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:04.592285 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 310/607] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:16:04 2022] PUT /identity/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/29048c2021e340b081ae44a390583e7a/roles/f57b3e871ce142a090d6aa7dea9ac7d5 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:16:04.593260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35515a9f-083a-4cf1-aefb-fee26916c972 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:04.691624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:04.707068 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f899cbd-9b63-47a6-9d1f-8ee43b48c136 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:04.734571 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35515a9f-083a-4cf1-aefb-fee26916c972 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:04.777565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c843501-3be3-4005-b10f-03802d785237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:04.778773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c843501-3be3-4005-b10f-03802d785237 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:04.779107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c843501-3be3-4005-b10f-03802d785237 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:04.779393 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c843501-3be3-4005-b10f-03802d785237 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:04.780392 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c843501-3be3-4005-b10f-03802d785237 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:04.780729 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c843501-3be3-4005-b10f-03802d785237 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:04.781950 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b83e38-4c0d-4cc4-9ab2-252542ae36d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:04.782742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c843501-3be3-4005-b10f-03802d785237 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:04.783223 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b83e38-4c0d-4cc4-9ab2-252542ae36d2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:04.783710 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b83e38-4c0d-4cc4-9ab2-252542ae36d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:04.784487 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b83e38-4c0d-4cc4-9ab2-252542ae36d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:04.794881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 311/608] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:16:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:04.800456 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b83e38-4c0d-4cc4-9ab2-252542ae36d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:04.800715 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c843501-3be3-4005-b10f-03802d785237 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:04.802519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b83e38-4c0d-4cc4-9ab2-252542ae36d2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:04.822841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-35b83e38-4c0d-4cc4-9ab2-252542ae36d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:04.822841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:04.822841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:04.822841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:04.822841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:04.822841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:04.822841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:04.822841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:04.822841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:04.822841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:04.822841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:04.822841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:04.822841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:04.822841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:04.822841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:04.823828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:04.823828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:04.823828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:04.823828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:04.823828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:04.823828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:04.823828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:04.823828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:04.823828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:04.823828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:04.823828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:04.823828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:04.823828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:04.823828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:04.823828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:04.823828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:04.823828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:04.824776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:04.824776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:04.824776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:04.824776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:04.824776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:04.824776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:04.824776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:04.824776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:04.828063 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 298/609] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:04.838553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b83e38-4c0d-4cc4-9ab2-252542ae36d2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:04.890147 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:04.963870 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c843501-3be3-4005-b10f-03802d785237 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:05.002881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b16176-59e1-4a16-9b12-22da728a7395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:05.003686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b16176-59e1-4a16-9b12-22da728a7395 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:05.003972 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b16176-59e1-4a16-9b12-22da728a7395 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:05.004249 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b16176-59e1-4a16-9b12-22da728a7395 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:05.017954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b16176-59e1-4a16-9b12-22da728a7395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:05.019450 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b16176-59e1-4a16-9b12-22da728a7395 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:05.039330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-09b16176-59e1-4a16-9b12-22da728a7395 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:05.039330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:05.039330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:05.039330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:05.039330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:05.039330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:05.039330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:05.039330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:05.039330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:05.039330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:05.039330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:05.039330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:05.039330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:05.039330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:05.039330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:05.040283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:05.040283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:05.040283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:05.040283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:05.040283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:05.040283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:05.040283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:05.040283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:05.040283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:05.040283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:05.040283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:05.040283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:05.040283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:05.040283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:05.040283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:05.040283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:05.040283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:05.041229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:05.041229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:05.041229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:05.041229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:05.044602 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 312/610] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:05.046422 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b16176-59e1-4a16-9b12-22da728a7395 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:05.094910 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b83e38-4c0d-4cc4-9ab2-252542ae36d2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:05.158042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:05.167946 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b16176-59e1-4a16-9b12-22da728a7395 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:05.171328 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17496695-aa3f-4f41-b253-d841bf187b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:05.171997 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17496695-aa3f-4f41-b253-d841bf187b24 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:05.172282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17496695-aa3f-4f41-b253-d841bf187b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:05.172581 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17496695-aa3f-4f41-b253-d841bf187b24 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:05.173381 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17496695-aa3f-4f41-b253-d841bf187b24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:05.173692 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17496695-aa3f-4f41-b253-d841bf187b24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:05.175147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17496695-aa3f-4f41-b253-d841bf187b24 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:05.199889 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 299/611] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:05.206638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17496695-aa3f-4f41-b253-d841bf187b24 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:05.239600 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510ac0e-83c5-42f4-a218-32361a7719b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:05.239844 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:05.240321 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510ac0e-83c5-42f4-a218-32361a7719b1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:05.240667 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510ac0e-83c5-42f4-a218-32361a7719b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:05.240950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6510ac0e-83c5-42f4-a218-32361a7719b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:05.241760 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510ac0e-83c5-42f4-a218-32361a7719b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:05.242069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510ac0e-83c5-42f4-a218-32361a7719b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:05.243457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6510ac0e-83c5-42f4-a218-32361a7719b1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:05.268853 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 313/612] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:05.274231 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510ac0e-83c5-42f4-a218-32361a7719b1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:05.372759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17496695-aa3f-4f41-b253-d841bf187b24 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:05.416405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf752f3-e21c-4d15-918a-5416db172c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:05.417346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf752f3-e21c-4d15-918a-5416db172c70 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:05.417673 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf752f3-e21c-4d15-918a-5416db172c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:05.417951 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf752f3-e21c-4d15-918a-5416db172c70 None admin] PATH_INFO: `/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/10d29b7934374f34923587eda9be0ed0/roles/f57b3e871ce142a090d6aa7dea9ac7d5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:05.447095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf752f3-e21c-4d15-918a-5416db172c70 None admin] RBAC: Authorizing `identity:create_grant(project_id=e71a3acd0adf46a2a35b965a27c4998f, user_id=10d29b7934374f34923587eda9be0ed0, role_id=f57b3e871ce142a090d6aa7dea9ac7d5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:05.449268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf752f3-e21c-4d15-918a-5416db172c70 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:05.462138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6510ac0e-83c5-42f4-a218-32361a7719b1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:05.471412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 300/613] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:16:05 2022] PUT /identity/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/10d29b7934374f34923587eda9be0ed0/roles/f57b3e871ce142a090d6aa7dea9ac7d5 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:16:05.509184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb2d82-5d80-4af5-b23c-e3e6ce89454c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:05.510012 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb2d82-5d80-4af5-b23c-e3e6ce89454c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:05.510423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb2d82-5d80-4af5-b23c-e3e6ce89454c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:05.510700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb2d82-5d80-4af5-b23c-e3e6ce89454c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:05.519105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb2d82-5d80-4af5-b23c-e3e6ce89454c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:05.520615 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb2d82-5d80-4af5-b23c-e3e6ce89454c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:05.536933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7ceb2d82-5d80-4af5-b23c-e3e6ce89454c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:16:05.536933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:05.536933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:05.536933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:05.536933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:05.536933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:05.536933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:05.536933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:05.536933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:05.536933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:05.536933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:05.536933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:05.536933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:05.536933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:05.536933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:05.536933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:05.537979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:05.537979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:05.537979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:05.537979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:05.537979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:05.537979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:05.537979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:05.537979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:05.537979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:05.537979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:05.537979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:05.537979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:05.537979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:05.537979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:05.537979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:05.537979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:05.537979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:05.537979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:05.539511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:05.539511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:05.539511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:05.539511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:05.539511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:05.539511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:05.539511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:05.539511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:05.539511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:16:05.539511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:05.539511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 314/614] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:16:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:05.544600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf752f3-e21c-4d15-918a-5416db172c70 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:05.557331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1202dfd-bf71-4c7b-93a4-ade6e53f3e51 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:05.557660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1202dfd-bf71-4c7b-93a4-ade6e53f3e51 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:05.557887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1202dfd-bf71-4c7b-93a4-ade6e53f3e51 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:05.559541 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 315/615] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:05.564902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3a570-5a40-4758-b644-ea1ddc206f90 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:05.565174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3a570-5a40-4758-b644-ea1ddc206f90 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:05.565456 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3a570-5a40-4758-b644-ea1ddc206f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:05.601182 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f8b3a570-5a40-4758-b644-ea1ddc206f90 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:05.603328 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f8b3a570-5a40-4758-b644-ea1ddc206f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:05.621926 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:05.690439 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:05.707823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 316/616] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:16:05.731805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf752f3-e21c-4d15-918a-5416db172c70 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:05.732980 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3a570-5a40-4758-b644-ea1ddc206f90 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:05.781349 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad658143-8c85-4522-88c7-7b772cd9a44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:05.782052 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad658143-8c85-4522-88c7-7b772cd9a44b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:05.782344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad658143-8c85-4522-88c7-7b772cd9a44b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:05.782691 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad658143-8c85-4522-88c7-7b772cd9a44b None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:05.783591 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad658143-8c85-4522-88c7-7b772cd9a44b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:05.783918 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad658143-8c85-4522-88c7-7b772cd9a44b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:05.785398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad658143-8c85-4522-88c7-7b772cd9a44b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:05.810335 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 301/617] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Jul 18 10:16:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:05.811879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad658143-8c85-4522-88c7-7b772cd9a44b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:05.897463 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3a570-5a40-4758-b644-ea1ddc206f90 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:05.944600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad658143-8c85-4522-88c7-7b772cd9a44b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:05.949914 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b19f58-466f-4aae-9d3d-48c8fc23c5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:05.950963 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b19f58-466f-4aae-9d3d-48c8fc23c5ec None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:05.954423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b19f58-466f-4aae-9d3d-48c8fc23c5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:05.955003 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b19f58-466f-4aae-9d3d-48c8fc23c5ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:05.978258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b19f58-466f-4aae-9d3d-48c8fc23c5ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:05.989096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-04b19f58-466f-4aae-9d3d-48c8fc23c5ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:05.989096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:05.989096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:05.989096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:05.989096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:05.989096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:05.989096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:05.989096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:05.989096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:05.989096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:05.989096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:05.989096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:05.989096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:05.989096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:05.989096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:05.989096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:05.990060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:05.990060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:05.990060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:05.990060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:05.990060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:05.990060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:05.990060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:05.990060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:05.990060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:05.990060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:05.990060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:05.990060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:05.990060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:05.990060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:05.990060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:05.990060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:05.990891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:05.990891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:05.990891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:05.990891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:05.990891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:05.990891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:05.990891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:05.990891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:05.990891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:05.990891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:05.990891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:05.990891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:05.998204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 317/618] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:16:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:06.009558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b19f58-466f-4aae-9d3d-48c8fc23c5ec None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:06.017255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:06.030098 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f57422-3d99-44c3-a3b7-6c5dd9b9ac89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:06.031266 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f57422-3d99-44c3-a3b7-6c5dd9b9ac89 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:06.031754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f57422-3d99-44c3-a3b7-6c5dd9b9ac89 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:06.032229 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f57422-3d99-44c3-a3b7-6c5dd9b9ac89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:06.044461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:06.055608 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f57422-3d99-44c3-a3b7-6c5dd9b9ac89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:06.058400 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f57422-3d99-44c3-a3b7-6c5dd9b9ac89 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:06.098093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-71f57422-3d99-44c3-a3b7-6c5dd9b9ac89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:06.098093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:06.098093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:06.098093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:06.098093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:06.098093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:06.098093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:06.098093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:06.098093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:06.098093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:06.098093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:06.098093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:06.098093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:06.098093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:06.098093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:06.099065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:06.099065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:06.099065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:06.099065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:06.099065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:06.099065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:06.099065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:06.099065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:06.099065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:06.099065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:06.099065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:06.099065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:06.099065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:06.099065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:06.099065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:06.099065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:06.099065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:06.100100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:06.100100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:06.100100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:06.100100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:06.109295 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 302/619] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:06.121826 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f57422-3d99-44c3-a3b7-6c5dd9b9ac89 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:06.218948 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b19f58-466f-4aae-9d3d-48c8fc23c5ec None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:06.283369 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c6ff6-4523-423c-afe6-f169bf446139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:06.284429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c6ff6-4523-423c-afe6-f169bf446139 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:06.284940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c6ff6-4523-423c-afe6-f169bf446139 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:06.285415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c6ff6-4523-423c-afe6-f169bf446139 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:06.286370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c6ff6-4523-423c-afe6-f169bf446139 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:06.287055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c6ff6-4523-423c-afe6-f169bf446139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:06.288878 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c6ff6-4523-423c-afe6-f169bf446139 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:06.297839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f57422-3d99-44c3-a3b7-6c5dd9b9ac89 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:06.307287 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 318/620] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:16:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:06.312477 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c6ff6-4523-423c-afe6-f169bf446139 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:06.370115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0eda1-6700-4e94-b514-352f9ec2f0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:06.371455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0eda1-6700-4e94-b514-352f9ec2f0bb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:06.372009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0eda1-6700-4e94-b514-352f9ec2f0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:06.372489 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0eda1-6700-4e94-b514-352f9ec2f0bb None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:06.373462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0eda1-6700-4e94-b514-352f9ec2f0bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:06.374756 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0eda1-6700-4e94-b514-352f9ec2f0bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:06.376350 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0eda1-6700-4e94-b514-352f9ec2f0bb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:06.394155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 303/621] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:06.395816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0eda1-6700-4e94-b514-352f9ec2f0bb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:06.438621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:06.466290 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:06.529046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c6ff6-4523-423c-afe6-f169bf446139 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:06.570366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0eda1-6700-4e94-b514-352f9ec2f0bb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:06.595728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb8ceb-1435-45f3-a28e-7b38ac9601f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:06.596924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb8ceb-1435-45f3-a28e-7b38ac9601f1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:06.597415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb8ceb-1435-45f3-a28e-7b38ac9601f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:06.597910 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb8ceb-1435-45f3-a28e-7b38ac9601f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:06.620252 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb8ceb-1435-45f3-a28e-7b38ac9601f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:06.622762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb8ceb-1435-45f3-a28e-7b38ac9601f1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:06.643604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b9d20-2447-4c48-9ca9-c2051565826e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:06.644421 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b9d20-2447-4c48-9ca9-c2051565826e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:06.644732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b9d20-2447-4c48-9ca9-c2051565826e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:06.645010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b9d20-2447-4c48-9ca9-c2051565826e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:06.646769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-5adb8ceb-1435-45f3-a28e-7b38ac9601f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:06.646769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:06.646769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:06.646769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:06.646769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:06.646769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:06.646769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:06.646769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:06.646769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:06.646769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:06.646769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:06.646769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:06.646769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:06.646769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:06.646769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:06.648202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:06.648202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:06.648202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:06.648202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:06.648202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:06.648202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:06.648202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:06.648202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:06.648202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:06.648202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:06.648202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:06.648202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:06.648202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:06.648202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:06.648202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:06.648202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:06.648202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:06.649157 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:06.649157 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:06.649157 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:06.649157 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:06.651466 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 319/622] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:06.657234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb8ceb-1435-45f3-a28e-7b38ac9601f1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:06.661064 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b9d20-2447-4c48-9ca9-c2051565826e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:06.662539 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b9d20-2447-4c48-9ca9-c2051565826e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:06.701083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4e5b9d20-2447-4c48-9ca9-c2051565826e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:06.701083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:06.701083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:06.701083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:06.701083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:06.701083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:06.701083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:06.701083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:06.701083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:06.701083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:06.701083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:06.701083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:06.701083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:06.701083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:06.701083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:06.702099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:06.702099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:06.702099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:06.702099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:06.702099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:06.702099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:06.702099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:06.702099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:06.702099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:06.702099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:06.702099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:06.702099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:06.702099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:06.702099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:06.702099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:06.702099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:06.702099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:06.703459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:06.703459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:06.703459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:06.703459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:06.703459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:06.703459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:06.703459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:06.703459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:06.703459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 304/623] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:06.713623 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b9d20-2447-4c48-9ca9-c2051565826e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:06.786708 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb8ceb-1435-45f3-a28e-7b38ac9601f1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:06.812528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:06.824863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:06.827106 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f365f-41d2-483f-9c7d-e8a24cca4094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:06.827815 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f365f-41d2-483f-9c7d-e8a24cca4094 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:06.828173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f365f-41d2-483f-9c7d-e8a24cca4094 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:06.828967 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f365f-41d2-483f-9c7d-e8a24cca4094 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:06.830807 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f365f-41d2-483f-9c7d-e8a24cca4094 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:06.831221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f365f-41d2-483f-9c7d-e8a24cca4094 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:06.832748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f365f-41d2-483f-9c7d-e8a24cca4094 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:06.833229 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b9d20-2447-4c48-9ca9-c2051565826e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:06.846235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 320/624] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:06.847928 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35871690-45ba-4fd8-8bfb-05bbf52812c8 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:06.848545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35871690-45ba-4fd8-8bfb-05bbf52812c8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:06.848820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35871690-45ba-4fd8-8bfb-05bbf52812c8 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:06.850664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 321/625] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:06.851657 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35871690-45ba-4fd8-8bfb-05bbf52812c8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:06.873184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd583db3-0589-4c06-bf09-8e200a10456e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:06.873844 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd583db3-0589-4c06-bf09-8e200a10456e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:06.874125 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd583db3-0589-4c06-bf09-8e200a10456e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:06.874400 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd583db3-0589-4c06-bf09-8e200a10456e None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:06.875185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd583db3-0589-4c06-bf09-8e200a10456e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:06.875498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd583db3-0589-4c06-bf09-8e200a10456e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:06.876966 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd583db3-0589-4c06-bf09-8e200a10456e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:06.890303 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 305/626] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:06.891643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe11468-8613-4954-83b6-8a0b78cb3ba5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:06.891939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe11468-8613-4954-83b6-8a0b78cb3ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:06.892213 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe11468-8613-4954-83b6-8a0b78cb3ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:06.908484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3fe11468-8613-4954-83b6-8a0b78cb3ba5 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:06.909700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3fe11468-8613-4954-83b6-8a0b78cb3ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:06.973761 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35871690-45ba-4fd8-8bfb-05bbf52812c8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:06.984352 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 306/627] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:16:06.985607 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe11468-8613-4954-83b6-8a0b78cb3ba5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:07.019923 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968d1ee-78f3-46f3-afde-4c99090012c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:07.021372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968d1ee-78f3-46f3-afde-4c99090012c4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:07.022028 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968d1ee-78f3-46f3-afde-4c99090012c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:07.022431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968d1ee-78f3-46f3-afde-4c99090012c4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:07.030716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968d1ee-78f3-46f3-afde-4c99090012c4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:07.032276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968d1ee-78f3-46f3-afde-4c99090012c4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:07.048365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7968d1ee-78f3-46f3-afde-4c99090012c4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 10:16:07.048365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:07.048365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:07.048365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:07.048365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:07.048365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:07.048365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:07.048365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:07.048365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:07.048365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:07.048365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:07.048365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:07.048365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:07.048365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:07.048365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:07.048365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:07.049601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:07.049601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:07.049601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:07.049601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:07.049601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:07.049601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:07.049601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:07.049601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:07.049601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:07.049601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:07.049601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:07.049601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:07.049601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:07.049601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:07.049601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:07.049601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:07.049601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:07.049601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:07.055844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:07.055844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:07.055844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:07.055844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:07.055844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:07.055844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:07.055844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:07.055844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:07.055844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 10:16:07.055844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:07.055844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 322/628] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:16:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:07.055844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e09ad5-6807-4d85-8b24-eea5392f832a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:07.055844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e09ad5-6807-4d85-8b24-eea5392f832a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:07.056686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e09ad5-6807-4d85-8b24-eea5392f832a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:07.056686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 323/629] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:07.056686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e09ad5-6807-4d85-8b24-eea5392f832a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:07.080570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:07.095740 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe11468-8613-4954-83b6-8a0b78cb3ba5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:07.106323 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:07.133911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df6f53-c473-4974-ab73-98c4df83a9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:07.134983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df6f53-c473-4974-ab73-98c4df83a9cc None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:07.135369 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df6f53-c473-4974-ab73-98c4df83a9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:07.135741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df6f53-c473-4974-ab73-98c4df83a9cc None admin] PATH_INFO: `/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/fd6168873f2846c8af73b903bba17eb8/roles/f57b3e871ce142a090d6aa7dea9ac7d5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:07.158270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df6f53-c473-4974-ab73-98c4df83a9cc None admin] RBAC: Authorizing `identity:create_grant(project_id=e71a3acd0adf46a2a35b965a27c4998f, user_id=fd6168873f2846c8af73b903bba17eb8, role_id=f57b3e871ce142a090d6aa7dea9ac7d5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:07.160523 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df6f53-c473-4974-ab73-98c4df83a9cc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:07.171627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e09ad5-6807-4d85-8b24-eea5392f832a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:07.184597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 307/630] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:16:06 2022] PUT /identity/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/fd6168873f2846c8af73b903bba17eb8/roles/f57b3e871ce142a090d6aa7dea9ac7d5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:16:07.186037 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df6f53-c473-4974-ab73-98c4df83a9cc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:07.226969 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5abc5f-0d6a-4ee6-a810-d4b523cc457b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:07.227945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5abc5f-0d6a-4ee6-a810-d4b523cc457b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:07.228332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5abc5f-0d6a-4ee6-a810-d4b523cc457b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:07.228717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5abc5f-0d6a-4ee6-a810-d4b523cc457b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:07.241972 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5abc5f-0d6a-4ee6-a810-d4b523cc457b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:07.250568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ee5abc5f-0d6a-4ee6-a810-d4b523cc457b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:07.250568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:07.250568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:07.250568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:07.250568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:07.250568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:07.250568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:07.250568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:07.250568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:07.250568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:07.250568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:07.250568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:07.250568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:07.250568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:07.250568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:07.250568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:07.251642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:07.251642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:07.251642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:07.251642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:07.251642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:07.251642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:07.251642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:07.251642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:07.251642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:07.251642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:07.251642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:07.251642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:07.251642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:07.251642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:07.251642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:07.251642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:07.252578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:07.252578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:07.252578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:07.252578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:07.252578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:07.252578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:07.252578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:07.252578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:07.252578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:07.252578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:07.252578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:07.252578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:07.252578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 324/631] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:16:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:07.253992 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5c4c1-6143-46ce-a5bd-3ef660d8c9f7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:07.254385 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5c4c1-6143-46ce-a5bd-3ef660d8c9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:07.254900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5c4c1-6143-46ce-a5bd-3ef660d8c9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:07.274382 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-36c5c4c1-6143-46ce-a5bd-3ef660d8c9f7 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:07.275870 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-36c5c4c1-6143-46ce-a5bd-3ef660d8c9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:07.322242 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df6f53-c473-4974-ab73-98c4df83a9cc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:07.367040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6461af-9665-404e-a707-d0544427d262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:07.367719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6461af-9665-404e-a707-d0544427d262 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:07.368114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6461af-9665-404e-a707-d0544427d262 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:07.368391 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6461af-9665-404e-a707-d0544427d262 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:07.369328 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6461af-9665-404e-a707-d0544427d262 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:07.369640 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6461af-9665-404e-a707-d0544427d262 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:07.371084 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6461af-9665-404e-a707-d0544427d262 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:07.374171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:07.375073 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 325/632] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:16:07.375706 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5c4c1-6143-46ce-a5bd-3ef660d8c9f7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:07.393693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 308/633] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Jul 18 10:16:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:07.399786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6461af-9665-404e-a707-d0544427d262 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:07.460897 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:07.507919 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6461af-9665-404e-a707-d0544427d262 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:07.523044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5c4c1-6143-46ce-a5bd-3ef660d8c9f7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:07.554792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563346df-04a9-4444-93e9-5c596c8aa0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:07.555822 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563346df-04a9-4444-93e9-5c596c8aa0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:07.556376 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563346df-04a9-4444-93e9-5c596c8aa0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:07.556775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563346df-04a9-4444-93e9-5c596c8aa0c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:07.559074 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76d127-5e46-4fe1-99dd-922e5c4d435d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:07.560243 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76d127-5e46-4fe1-99dd-922e5c4d435d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:07.560731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76d127-5e46-4fe1-99dd-922e5c4d435d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:07.561158 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76d127-5e46-4fe1-99dd-922e5c4d435d None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:07.562162 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76d127-5e46-4fe1-99dd-922e5c4d435d None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:07.562680 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76d127-5e46-4fe1-99dd-922e5c4d435d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:07.564304 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76d127-5e46-4fe1-99dd-922e5c4d435d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:07.574211 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 326/634] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:16:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:07.575327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76d127-5e46-4fe1-99dd-922e5c4d435d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:07.635780 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:07.646853 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563346df-04a9-4444-93e9-5c596c8aa0c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:07.649128 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563346df-04a9-4444-93e9-5c596c8aa0c0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:07.676885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:07.683839 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76d127-5e46-4fe1-99dd-922e5c4d435d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:07.732533 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfe03b-c2e3-424c-8672-93a68c0eb0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:07.733884 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfe03b-c2e3-424c-8672-93a68c0eb0ac None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:07.734299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfe03b-c2e3-424c-8672-93a68c0eb0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:07.734586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfe03b-c2e3-424c-8672-93a68c0eb0ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:07.748646 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfe03b-c2e3-424c-8672-93a68c0eb0ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:07.760666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-33dfe03b-c2e3-424c-8672-93a68c0eb0ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:07.760666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:07.760666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:07.760666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:07.760666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:07.760666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:07.760666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:07.760666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:07.760666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:07.760666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:07.760666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:07.760666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:07.760666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:07.760666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:07.760666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:07.760666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:07.761864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:07.761864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:07.761864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:07.761864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:07.761864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:07.761864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:07.761864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:07.761864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:07.761864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:07.761864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:07.761864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:07.761864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:07.761864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:07.761864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:07.761864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:07.761864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:07.762811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:07.762811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:07.762811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:07.762811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:07.762811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:07.762811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:07.762811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:07.762811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:07.762811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:07.762811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:07.762811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:07.762811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:07.764884 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 327/635] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:16:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:07.765494 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfe03b-c2e3-424c-8672-93a68c0eb0ac None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:07.782511 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 309/636] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:16:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:16:07.783581 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563346df-04a9-4444-93e9-5c596c8aa0c0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:07.871364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfe03b-c2e3-424c-8672-93a68c0eb0ac None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:07.891015 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:07.899635 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563346df-04a9-4444-93e9-5c596c8aa0c0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:07.906573 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e73d5-95e7-40e1-84de-ed1f582c44ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:07.907417 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e73d5-95e7-40e1-84de-ed1f582c44ab None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:07.907656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e73d5-95e7-40e1-84de-ed1f582c44ab None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:07.907929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e73d5-95e7-40e1-84de-ed1f582c44ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:07.919166 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e73d5-95e7-40e1-84de-ed1f582c44ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:07.920652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e73d5-95e7-40e1-84de-ed1f582c44ab None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:07.935834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c3465-ca0b-4aa6-86bc-84f1e3b5fda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:07.936786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c3465-ca0b-4aa6-86bc-84f1e3b5fda5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:07.937263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c3465-ca0b-4aa6-86bc-84f1e3b5fda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:07.937714 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c3465-ca0b-4aa6-86bc-84f1e3b5fda5 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:07.937825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9d3e73d5-95e7-40e1-84de-ed1f582c44ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:07.937825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:07.937825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:07.937825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:07.937825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:07.937825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:07.937825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:07.937825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:07.937825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:07.937825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:07.937825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:07.937825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:07.937825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:07.937825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:07.937825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:07.938739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:07.938739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:07.938739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:07.938739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:07.938739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:07.938739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:07.938739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:07.938739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:07.938739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:07.938739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:07.938739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:07.938739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:07.938739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:07.938739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:07.938739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:07.938739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:07.938739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:07.939635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:07.939635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:07.939635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:07.939635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:07.939635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 328/637] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:07.939932 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e73d5-95e7-40e1-84de-ed1f582c44ab None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:07.941207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c3465-ca0b-4aa6-86bc-84f1e3b5fda5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:07.941580 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c3465-ca0b-4aa6-86bc-84f1e3b5fda5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:07.943007 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c3465-ca0b-4aa6-86bc-84f1e3b5fda5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:07.955479 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:07.957076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 310/638] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:16:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:07.958502 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cffd5e-4945-48c2-9881-ff51272d8cce None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:07.958855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cffd5e-4945-48c2-9881-ff51272d8cce None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:07.959120 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cffd5e-4945-48c2-9881-ff51272d8cce None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:07.964763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 311/639] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:07.964763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cffd5e-4945-48c2-9881-ff51272d8cce None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:08.051804 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e73d5-95e7-40e1-84de-ed1f582c44ab None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:08.070247 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cffd5e-4945-48c2-9881-ff51272d8cce None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:08.086514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4880e-5b89-4db2-a7e1-48328dbc3b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:08.087415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4880e-5b89-4db2-a7e1-48328dbc3b96 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:08.087805 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4880e-5b89-4db2-a7e1-48328dbc3b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:08.088083 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4880e-5b89-4db2-a7e1-48328dbc3b96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:08.098463 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4880e-5b89-4db2-a7e1-48328dbc3b96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:08.099906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4880e-5b89-4db2-a7e1-48328dbc3b96 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:08.108092 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30543c90-f78a-4ebc-8dd6-a8447b077c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:08.108985 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30543c90-f78a-4ebc-8dd6-a8447b077c84 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:08.109149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30543c90-f78a-4ebc-8dd6-a8447b077c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:08.109403 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30543c90-f78a-4ebc-8dd6-a8447b077c84 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:08.110122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30543c90-f78a-4ebc-8dd6-a8447b077c84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:08.110433 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30543c90-f78a-4ebc-8dd6-a8447b077c84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:08.111789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30543c90-f78a-4ebc-8dd6-a8447b077c84 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:08.118472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-41f4880e-5b89-4db2-a7e1-48328dbc3b96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:08.118472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:08.118472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:08.118472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:08.118472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:08.118472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:08.118472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:08.118472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:08.118472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:08.118472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:08.118472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:08.118472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:08.118472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:08.118472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:08.118472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:08.119569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:08.119569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:08.119569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:08.119569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:08.119569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:08.119569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:08.119569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:08.119569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:08.119569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:08.119569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:08.119569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:08.119569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:08.119569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:08.119569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:08.119569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:08.119569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:08.119569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:08.121274 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:08.121274 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:08.121274 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:08.121274 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:08.128717 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 312/640] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:08.128717 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30543c90-f78a-4ebc-8dd6-a8447b077c84 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:08.128953 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 329/641] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:08.128953 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39658083-9c20-4648-8ed4-4c6352197312 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:08.128953 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39658083-9c20-4648-8ed4-4c6352197312 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:08.128953 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39658083-9c20-4648-8ed4-4c6352197312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:08.144599 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-39658083-9c20-4648-8ed4-4c6352197312 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:08.145932 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-39658083-9c20-4648-8ed4-4c6352197312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:08.147166 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:08.217906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:08.236591 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 330/642] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:16:08.236591 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39658083-9c20-4648-8ed4-4c6352197312 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:08.240814 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30543c90-f78a-4ebc-8dd6-a8447b077c84 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:08.280906 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f107a-e66f-4741-97ac-87f681881dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:08.281825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f107a-e66f-4741-97ac-87f681881dea None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:08.282178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f107a-e66f-4741-97ac-87f681881dea None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:08.282520 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f107a-e66f-4741-97ac-87f681881dea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:08.293902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f107a-e66f-4741-97ac-87f681881dea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:08.295415 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f107a-e66f-4741-97ac-87f681881dea None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:08.313649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a25f107a-e66f-4741-97ac-87f681881dea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:08.313649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:08.313649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:08.313649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:08.313649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:08.313649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:08.313649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:08.313649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:08.313649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:08.313649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:08.313649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:08.313649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:08.313649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:08.313649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:08.313649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:08.314832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:08.314832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:08.314832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:08.314832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:08.314832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:08.314832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:08.314832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:08.314832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:08.314832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:08.314832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:08.314832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:08.314832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:08.314832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:08.314832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:08.314832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:08.314832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:08.314832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:08.320911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:08.320911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:08.320911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:08.320911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:08.320911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 313/643] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:08.320911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f107a-e66f-4741-97ac-87f681881dea None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:08.353953 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39658083-9c20-4648-8ed4-4c6352197312 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:08.388950 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:08.394973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa686ed1-1398-4639-9895-f8ec607d7a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:08.395919 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa686ed1-1398-4639-9895-f8ec607d7a5a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:08.396431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa686ed1-1398-4639-9895-f8ec607d7a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:08.396933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa686ed1-1398-4639-9895-f8ec607d7a5a None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:08.398435 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa686ed1-1398-4639-9895-f8ec607d7a5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:08.398954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa686ed1-1398-4639-9895-f8ec607d7a5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:08.400719 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa686ed1-1398-4639-9895-f8ec607d7a5a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:08.414788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 331/644] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:08.415737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa686ed1-1398-4639-9895-f8ec607d7a5a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:08.429381 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f107a-e66f-4741-97ac-87f681881dea None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:08.464262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47295e-78bc-4d32-837f-e21af6d660ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:08.465091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47295e-78bc-4d32-837f-e21af6d660ec None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:08.465380 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47295e-78bc-4d32-837f-e21af6d660ec None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:08.465661 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47295e-78bc-4d32-837f-e21af6d660ec None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:08.474175 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47295e-78bc-4d32-837f-e21af6d660ec None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:08.474663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:08.477180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47295e-78bc-4d32-837f-e21af6d660ec None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:08.494499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ca47295e-78bc-4d32-837f-e21af6d660ec None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:16:08.494499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:08.494499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:08.494499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:08.494499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:08.494499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:08.494499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:08.494499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:08.494499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:08.494499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:08.494499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:08.494499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:08.494499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:08.494499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:08.494499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:08.494499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:08.496270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:08.496270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:08.496270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:08.496270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:08.496270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:08.496270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:08.496270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:08.496270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:08.496270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:08.496270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:08.496270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:08.496270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:08.496270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:08.496270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:08.496270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:08.496270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:08.496270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:08.496270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:08.497434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:08.497434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:08.497434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:08.497434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:08.497434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:08.497434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:08.497434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:08.497434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:08.497434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:16:08.497434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:08.497434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 314/645] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:16:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:08.497434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47295e-78bc-4d32-837f-e21af6d660ec None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:08.537020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa686ed1-1398-4639-9895-f8ec607d7a5a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:08.576344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5581f-5ee3-45dc-bb21-dd386e2b44bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:08.577847 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5581f-5ee3-45dc-bb21-dd386e2b44bd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:08.578340 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5581f-5ee3-45dc-bb21-dd386e2b44bd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:08.578773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5581f-5ee3-45dc-bb21-dd386e2b44bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:08.590547 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5581f-5ee3-45dc-bb21-dd386e2b44bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:08.598258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-45b5581f-5ee3-45dc-bb21-dd386e2b44bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:08.598258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:08.598258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:08.598258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:08.598258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:08.598258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:08.598258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:08.598258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:08.598258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:08.598258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:08.598258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:08.598258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:08.598258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:08.598258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:08.598258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:08.598258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:08.599191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:08.599191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:08.599191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:08.599191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:08.599191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:08.599191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:08.599191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:08.599191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:08.599191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:08.599191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:08.599191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:08.599191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:08.599191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:08.599191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:08.599191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:08.599191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:08.600053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:08.600053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:08.600053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:08.600053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:08.600053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:08.600053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:08.600053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:08.600053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:08.600053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:08.600053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:08.600053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:08.600053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:08.600053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 332/646] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:16:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:08.600832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5581f-5ee3-45dc-bb21-dd386e2b44bd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:08.610157 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47295e-78bc-4d32-837f-e21af6d660ec None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:08.637578 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:08.646315 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f1fe0-b9a0-4778-93de-7f46d73f8036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:08.646920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f1fe0-b9a0-4778-93de-7f46d73f8036 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:08.647207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f1fe0-b9a0-4778-93de-7f46d73f8036 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:08.647525 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f1fe0-b9a0-4778-93de-7f46d73f8036 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:08.648237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f1fe0-b9a0-4778-93de-7f46d73f8036 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:08.648570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f1fe0-b9a0-4778-93de-7f46d73f8036 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:08.649938 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f1fe0-b9a0-4778-93de-7f46d73f8036 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:08.666495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 315/647] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:08.666495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f1fe0-b9a0-4778-93de-7f46d73f8036 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:08.722022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5581f-5ee3-45dc-bb21-dd386e2b44bd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:08.733818 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:08.758695 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd94522-ee07-478e-82f4-3b9c1ee6010f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:08.759494 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd94522-ee07-478e-82f4-3b9c1ee6010f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:08.759778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd94522-ee07-478e-82f4-3b9c1ee6010f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:08.760048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd94522-ee07-478e-82f4-3b9c1ee6010f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:08.766833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd94522-ee07-478e-82f4-3b9c1ee6010f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:08.768280 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd94522-ee07-478e-82f4-3b9c1ee6010f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:08.770944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f1fe0-b9a0-4778-93de-7f46d73f8036 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:08.782748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-4dd94522-ee07-478e-82f4-3b9c1ee6010f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:08.782748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:08.782748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:08.782748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:08.782748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:08.782748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:08.782748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:08.782748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:08.782748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:08.782748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:08.782748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:08.782748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:08.782748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:08.782748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:08.782748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:08.783865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:08.783865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:08.783865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:08.783865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:08.783865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:08.783865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:08.783865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:08.783865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:08.783865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:08.783865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:08.783865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:08.783865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:08.783865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:08.783865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:08.783865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:08.783865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:08.783865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:08.784889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:08.784889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:08.784889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:08.784889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:08.784889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:08.784889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:08.784889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:08.784889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:08.784889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 333/648] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:08.784889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd94522-ee07-478e-82f4-3b9c1ee6010f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:08.814617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddec320-e7d7-455e-b968-296e23871bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:08.815286 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddec320-e7d7-455e-b968-296e23871bac None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:08.815571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddec320-e7d7-455e-b968-296e23871bac None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:08.815844 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddec320-e7d7-455e-b968-296e23871bac None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:08.816711 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddec320-e7d7-455e-b968-296e23871bac None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:08.817018 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddec320-e7d7-455e-b968-296e23871bac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:08.818439 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddec320-e7d7-455e-b968-296e23871bac None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:08.836990 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 316/649] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Jul 18 10:16:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:08.841561 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddec320-e7d7-455e-b968-296e23871bac None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:08.886766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:08.906009 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd94522-ee07-478e-82f4-3b9c1ee6010f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:08.946403 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29ea51-26e5-483c-a607-8be620511870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:08.947362 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29ea51-26e5-483c-a607-8be620511870 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:08.947683 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddec320-e7d7-455e-b968-296e23871bac None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:08.948302 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29ea51-26e5-483c-a607-8be620511870 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:08.948748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29ea51-26e5-483c-a607-8be620511870 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:08.949637 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29ea51-26e5-483c-a607-8be620511870 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:08.950100 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29ea51-26e5-483c-a607-8be620511870 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:08.951477 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29ea51-26e5-483c-a607-8be620511870 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:08.964114 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 334/650] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:16:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:08.964114 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29ea51-26e5-483c-a607-8be620511870 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:08.988243 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889d512-d18e-45c2-84c0-9aa0fd21f5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:08.989069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889d512-d18e-45c2-84c0-9aa0fd21f5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:08.989350 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889d512-d18e-45c2-84c0-9aa0fd21f5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:08.989622 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889d512-d18e-45c2-84c0-9aa0fd21f5f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:08.997905 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889d512-d18e-45c2-84c0-9aa0fd21f5f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:08.999332 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889d512-d18e-45c2-84c0-9aa0fd21f5f1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:09.001238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:09.016183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0889d512-d18e-45c2-84c0-9aa0fd21f5f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:16:09.016183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:09.016183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:09.016183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:09.016183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:09.016183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:09.016183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:09.016183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:09.016183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:09.016183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:09.016183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:09.016183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:09.016183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:09.016183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:09.016183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:09.016183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:09.017261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:09.017261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:09.017261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:09.017261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:09.017261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:09.017261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:09.017261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:09.017261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:09.017261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:09.017261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:09.017261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:09.017261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:09.017261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:09.017261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:09.017261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:09.017261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:09.017261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:09.017261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:09.018191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:09.018191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:09.018191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:09.018191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:09.018191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:09.018191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:09.018191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:09.018191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:09.018191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:16:09.018191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:09.018191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 317/651] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jul 18 10:16:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:09.018191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889d512-d18e-45c2-84c0-9aa0fd21f5f1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:09.079490 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29ea51-26e5-483c-a607-8be620511870 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:09.116294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cadef-0e95-40e6-a3a1-45f80175f9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:09.117236 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cadef-0e95-40e6-a3a1-45f80175f9bd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:09.117522 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cadef-0e95-40e6-a3a1-45f80175f9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:09.117949 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cadef-0e95-40e6-a3a1-45f80175f9bd None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:09.119059 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cadef-0e95-40e6-a3a1-45f80175f9bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:09.119640 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cadef-0e95-40e6-a3a1-45f80175f9bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:09.121277 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cadef-0e95-40e6-a3a1-45f80175f9bd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:09.124159 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889d512-d18e-45c2-84c0-9aa0fd21f5f1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:09.131770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:09.142922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 335/652] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:09.143551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cadef-0e95-40e6-a3a1-45f80175f9bd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:09.167245 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886a190-4a41-4913-ad0b-dbb01c326374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:09.167833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886a190-4a41-4913-ad0b-dbb01c326374 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:09.168116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886a190-4a41-4913-ad0b-dbb01c326374 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:09.168384 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886a190-4a41-4913-ad0b-dbb01c326374 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:09.241856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:09.249538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886a190-4a41-4913-ad0b-dbb01c326374 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:09.251005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886a190-4a41-4913-ad0b-dbb01c326374 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:09.253336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cadef-0e95-40e6-a3a1-45f80175f9bd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:09.287830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f2ba6-a9f3-4dc4-bd3d-308e81a84f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:09.289189 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f2ba6-a9f3-4dc4-bd3d-308e81a84f7d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:09.289479 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f2ba6-a9f3-4dc4-bd3d-308e81a84f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:09.289757 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f2ba6-a9f3-4dc4-bd3d-308e81a84f7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:09.299707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f2ba6-a9f3-4dc4-bd3d-308e81a84f7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:09.301212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f2ba6-a9f3-4dc4-bd3d-308e81a84f7d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:09.318653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-943f2ba6-a9f3-4dc4-bd3d-308e81a84f7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:09.318653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:09.318653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:09.318653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:09.318653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:09.318653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:09.318653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:09.318653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:09.318653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:09.318653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:09.318653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:09.318653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:09.318653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:09.318653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:09.318653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:09.319607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:09.319607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:09.319607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:09.319607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:09.319607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:09.319607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:09.319607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:09.319607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:09.319607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:09.319607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:09.319607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:09.319607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:09.319607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:09.319607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:09.319607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:09.319607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:09.319607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:09.320493 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:09.320493 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:09.320493 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:09.320493 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:09.320493 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 336/653] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:09.320813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f2ba6-a9f3-4dc4-bd3d-308e81a84f7d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:09.353272 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 318/654] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:16:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:16:09.354122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886a190-4a41-4913-ad0b-dbb01c326374 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:09.362595 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:09.419590 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f2ba6-a9f3-4dc4-bd3d-308e81a84f7d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:09.452416 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b523296-0f21-4175-9311-27efa47eff22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:09.453102 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b523296-0f21-4175-9311-27efa47eff22 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:09.453385 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b523296-0f21-4175-9311-27efa47eff22 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:09.453661 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b523296-0f21-4175-9311-27efa47eff22 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:09.454495 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b523296-0f21-4175-9311-27efa47eff22 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:09.454805 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b523296-0f21-4175-9311-27efa47eff22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:09.454960 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886a190-4a41-4913-ad0b-dbb01c326374 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:09.456229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b523296-0f21-4175-9311-27efa47eff22 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:09.473346 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 337/655] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jul 18 10:16:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:09.474568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e3e2f-c078-4215-a276-4374c61e2ff1 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:09.474855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e3e2f-c078-4215-a276-4374c61e2ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:09.475125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e3e2f-c078-4215-a276-4374c61e2ff1 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:09.476569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 338/656] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:09.477263 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e3e2f-c078-4215-a276-4374c61e2ff1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:09.480083 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:09.493500 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be068a9-e2e7-4ad5-887a-3018402c048a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:09.494162 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be068a9-e2e7-4ad5-887a-3018402c048a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:09.494449 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be068a9-e2e7-4ad5-887a-3018402c048a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:09.494727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be068a9-e2e7-4ad5-887a-3018402c048a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:09.495558 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be068a9-e2e7-4ad5-887a-3018402c048a None admin] RBAC: Adding query filter params (role.id=f57b3e871ce142a090d6aa7dea9ac7d5, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=3cee4d23797e430e9eeddc394ed7f50e) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:09.495872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be068a9-e2e7-4ad5-887a-3018402c048a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:09.497355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be068a9-e2e7-4ad5-887a-3018402c048a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:09.503182 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 319/657] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:16:09 2022] GET /identity/v3/role_assignments?user.id=3cee4d23797e430e9eeddc394ed7f50e&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=f57b3e871ce142a090d6aa7dea9ac7d5 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:09.503955 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be068a9-e2e7-4ad5-887a-3018402c048a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:09.581324 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:09.598022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e3e2f-c078-4215-a276-4374c61e2ff1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:09.617036 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be068a9-e2e7-4ad5-887a-3018402c048a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:09.636592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162ef08-ea8a-45f2-b79f-d4d5b9d9e878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:09.637281 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162ef08-ea8a-45f2-b79f-d4d5b9d9e878 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:09.637569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162ef08-ea8a-45f2-b79f-d4d5b9d9e878 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:09.637848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162ef08-ea8a-45f2-b79f-d4d5b9d9e878 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:09.638593 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162ef08-ea8a-45f2-b79f-d4d5b9d9e878 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:09.638905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162ef08-ea8a-45f2-b79f-d4d5b9d9e878 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:09.640322 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162ef08-ea8a-45f2-b79f-d4d5b9d9e878 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:09.653454 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 339/658] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:09.653657 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162ef08-ea8a-45f2-b79f-d4d5b9d9e878 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:09.659457 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3f1ab-35fd-40dd-a25b-43cd50c3f8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:09.660255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3f1ab-35fd-40dd-a25b-43cd50c3f8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:09.660561 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3f1ab-35fd-40dd-a25b-43cd50c3f8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:09.660845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3f1ab-35fd-40dd-a25b-43cd50c3f8a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:09.673755 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3f1ab-35fd-40dd-a25b-43cd50c3f8a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:09.675210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3f1ab-35fd-40dd-a25b-43cd50c3f8a9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:09.693412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a4c3f1ab-35fd-40dd-a25b-43cd50c3f8a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:09.693412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:09.693412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:09.693412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:09.693412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:09.693412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:09.693412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:09.693412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:09.693412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:09.693412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:09.693412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:09.693412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:09.693412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:09.693412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:09.693412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:09.694632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:09.694632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:09.694632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:09.694632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:09.694632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:09.694632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:09.694632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:09.694632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:09.694632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:09.694632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:09.694632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:09.694632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:09.694632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:09.694632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:09.694632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:09.694632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:09.694632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:09.695805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:09.695805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:09.695805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:09.695805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:09.696346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 320/659] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:09.697574 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f062d-44b7-4898-842a-28876e36ed0e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:09.697856 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f062d-44b7-4898-842a-28876e36ed0e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:09.698138 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f062d-44b7-4898-842a-28876e36ed0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:09.715310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-504f062d-44b7-4898-842a-28876e36ed0e None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:09.716529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-504f062d-44b7-4898-842a-28876e36ed0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:09.732692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:09.776058 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162ef08-ea8a-45f2-b79f-d4d5b9d9e878 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:09.789983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 321/660] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:16:09.790857 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f062d-44b7-4898-842a-28876e36ed0e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:09.821557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc996e54-ace8-4970-a0ae-4390be61becd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:09.822213 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc996e54-ace8-4970-a0ae-4390be61becd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:09.822502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc996e54-ace8-4970-a0ae-4390be61becd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:09.822782 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc996e54-ace8-4970-a0ae-4390be61becd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:09.859539 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:09.893836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc996e54-ace8-4970-a0ae-4390be61becd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:09.895323 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc996e54-ace8-4970-a0ae-4390be61becd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:09.910237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f062d-44b7-4898-842a-28876e36ed0e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:09.948188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa58689-8b6f-452e-b695-2ce275774624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:09.948979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa58689-8b6f-452e-b695-2ce275774624 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:09.949230 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa58689-8b6f-452e-b695-2ce275774624 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:09.949508 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa58689-8b6f-452e-b695-2ce275774624 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:09.958059 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa58689-8b6f-452e-b695-2ce275774624 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:09.958733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:09.960863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa58689-8b6f-452e-b695-2ce275774624 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:09.976416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4fa58689-8b6f-452e-b695-2ce275774624 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 10:16:09.976416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:09.976416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:09.976416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:09.976416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:09.976416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:09.976416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:09.976416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:09.976416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:09.976416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:09.976416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:09.976416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:09.976416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:09.976416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:09.976416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:09.976416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:09.977555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:09.977555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:09.977555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:09.977555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:09.977555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:09.977555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:09.977555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:09.977555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:09.977555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:09.977555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:09.977555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:09.977555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:09.977555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:09.977555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:09.977555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:09.977555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:09.977555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:09.977555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:09.978507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:09.978507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:09.978507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:09.978507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:09.978507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:09.978507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:09.978507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:09.978507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:09.978507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 10:16:09.978507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:09.978507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 322/661] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:16:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:09.979118 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa58689-8b6f-452e-b695-2ce275774624 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:10.005431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 340/662] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:16:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:16:10.006569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc996e54-ace8-4970-a0ae-4390be61becd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:10.104436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa58689-8b6f-452e-b695-2ce275774624 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:10.110854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc996e54-ace8-4970-a0ae-4390be61becd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:10.120917 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:10.144497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc81c4d-81e7-411b-99e9-55dbcb98552f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:10.145104 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8d3bf-a7fe-4395-b75e-d9f17884b8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:10.145241 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc81c4d-81e7-411b-99e9-55dbcb98552f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:10.145409 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc81c4d-81e7-411b-99e9-55dbcb98552f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:10.145681 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc81c4d-81e7-411b-99e9-55dbcb98552f None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:10.145888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8d3bf-a7fe-4395-b75e-d9f17884b8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:10.146174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8d3bf-a7fe-4395-b75e-d9f17884b8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:10.146403 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc81c4d-81e7-411b-99e9-55dbcb98552f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:10.146500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8d3bf-a7fe-4395-b75e-d9f17884b8e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:10.146711 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc81c4d-81e7-411b-99e9-55dbcb98552f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:10.148224 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc81c4d-81e7-411b-99e9-55dbcb98552f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:10.156848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8d3bf-a7fe-4395-b75e-d9f17884b8e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:10.164863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 323/663] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:10.165164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f1a8d3bf-a7fe-4395-b75e-d9f17884b8e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:10.165164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:10.165164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:10.165164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:10.165164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:10.165164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:10.165164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:10.165164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:10.165164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:10.165164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:10.165164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:10.165164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:10.165164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:10.165164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:10.165164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:10.165164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:10.166151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:10.166151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:10.166151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:10.166151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:10.166151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:10.166151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:10.166151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:10.166151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:10.166151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:10.166151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:10.166151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:10.166151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:10.166151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:10.166151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:10.166151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:10.166151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:10.167351 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:10.167351 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:10.167351 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:10.167351 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:10.167351 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:10.167351 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:10.167351 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:10.167351 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:10.167351 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:10.167351 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:10.167351 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:10.167351 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:10.167351 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 341/664] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:16:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:10.168286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8d3bf-a7fe-4395-b75e-d9f17884b8e0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:10.169538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc81c4d-81e7-411b-99e9-55dbcb98552f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:10.197078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:10.278688 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8d3bf-a7fe-4395-b75e-d9f17884b8e0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:10.286284 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc81c4d-81e7-411b-99e9-55dbcb98552f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:10.314096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d26b9a-2ed0-4244-aebc-d2b05abf8cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:10.314893 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d26b9a-2ed0-4244-aebc-d2b05abf8cef None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:10.315185 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d26b9a-2ed0-4244-aebc-d2b05abf8cef None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:10.315461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d26b9a-2ed0-4244-aebc-d2b05abf8cef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:10.326614 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d26b9a-2ed0-4244-aebc-d2b05abf8cef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:10.328077 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d26b9a-2ed0-4244-aebc-d2b05abf8cef None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:10.329009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96384042-d207-4f25-a062-0e99578ed9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:10.330064 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96384042-d207-4f25-a062-0e99578ed9ce None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:10.330843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96384042-d207-4f25-a062-0e99578ed9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:10.331337 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96384042-d207-4f25-a062-0e99578ed9ce None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:10.332378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96384042-d207-4f25-a062-0e99578ed9ce None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:10.332867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96384042-d207-4f25-a062-0e99578ed9ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:10.334965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96384042-d207-4f25-a062-0e99578ed9ce None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:10.344780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-16d26b9a-2ed0-4244-aebc-d2b05abf8cef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:10.344780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:10.344780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:10.344780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:10.344780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:10.344780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:10.344780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:10.344780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:10.344780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:10.344780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:10.344780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:10.344780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:10.344780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:10.344780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:10.344780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:10.345855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:10.345855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:10.345855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:10.345855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:10.345855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:10.345855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:10.345855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:10.345855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:10.345855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:10.345855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:10.345855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:10.345855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:10.345855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:10.345855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:10.345855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:10.345855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:10.345855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:10.346764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:10.346764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:10.346764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:10.346764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:10.346764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 342/665] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:10.346764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d26b9a-2ed0-4244-aebc-d2b05abf8cef None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:10.364271 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 324/666] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Jul 18 10:16:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:10.364271 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96384042-d207-4f25-a062-0e99578ed9ce None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:10.397751 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:10.435496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:10.462243 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d26b9a-2ed0-4244-aebc-d2b05abf8cef None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:10.473928 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96384042-d207-4f25-a062-0e99578ed9ce None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:10.500044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890283e1-cce6-4e6f-9af9-0bf7395be8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:10.500864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890283e1-cce6-4e6f-9af9-0bf7395be8e8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:10.501146 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890283e1-cce6-4e6f-9af9-0bf7395be8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:10.501416 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890283e1-cce6-4e6f-9af9-0bf7395be8e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:10.509031 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890283e1-cce6-4e6f-9af9-0bf7395be8e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:10.510752 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890283e1-cce6-4e6f-9af9-0bf7395be8e8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:10.511621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d54d4-7470-401c-bf74-2bf41815c3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:10.512394 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d54d4-7470-401c-bf74-2bf41815c3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:10.512814 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d54d4-7470-401c-bf74-2bf41815c3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:10.513134 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d54d4-7470-401c-bf74-2bf41815c3b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:10.514560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d54d4-7470-401c-bf74-2bf41815c3b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:10.515716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d54d4-7470-401c-bf74-2bf41815c3b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:10.516739 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d54d4-7470-401c-bf74-2bf41815c3b2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:10.531600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 325/667] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:16:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:10.531600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d54d4-7470-401c-bf74-2bf41815c3b2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:10.531854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-890283e1-cce6-4e6f-9af9-0bf7395be8e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:10.531854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:10.531854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:10.531854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:10.531854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:10.531854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:10.531854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:10.531854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:10.531854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:10.531854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:10.531854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:10.531854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:10.531854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:10.531854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:10.531854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:10.532731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:10.532731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:10.532731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:10.532731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:10.532731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:10.532731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:10.532731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:10.532731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:10.532731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:10.532731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:10.532731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:10.532731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:10.532731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:10.532731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:10.532731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:10.532731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:10.532731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:10.533642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:10.533642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:10.533642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:10.533642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:10.533642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:10.533642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:10.533642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:10.533642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:10.533642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 343/668] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:10.533642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890283e1-cce6-4e6f-9af9-0bf7395be8e8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:10.633517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:10.644045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890283e1-cce6-4e6f-9af9-0bf7395be8e8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:10.644860 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d54d4-7470-401c-bf74-2bf41815c3b2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:10.682577 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f3ab9-5db5-4733-9c77-f70e8e7841df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:10.682870 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6c8bb-3b50-4b9a-8004-f74616a6e03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:10.683234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f3ab9-5db5-4733-9c77-f70e8e7841df None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:10.683519 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f3ab9-5db5-4733-9c77-f70e8e7841df None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:10.683802 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f3ab9-5db5-4733-9c77-f70e8e7841df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:10.684195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6c8bb-3b50-4b9a-8004-f74616a6e03c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:10.684789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6c8bb-3b50-4b9a-8004-f74616a6e03c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:10.685208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6c8bb-3b50-4b9a-8004-f74616a6e03c None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:10.686095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6c8bb-3b50-4b9a-8004-f74616a6e03c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:10.686562 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6c8bb-3b50-4b9a-8004-f74616a6e03c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:10.686700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:10.687943 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6c8bb-3b50-4b9a-8004-f74616a6e03c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:10.701107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 326/669] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:10.701937 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6c8bb-3b50-4b9a-8004-f74616a6e03c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:10.769352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f3ab9-5db5-4733-9c77-f70e8e7841df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:10.770924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f3ab9-5db5-4733-9c77-f70e8e7841df None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:10.809131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6c8bb-3b50-4b9a-8004-f74616a6e03c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:10.844955 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d97a6-d5b9-432a-a949-a0e206ffa545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:10.845780 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d97a6-d5b9-432a-a949-a0e206ffa545 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:10.846069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d97a6-d5b9-432a-a949-a0e206ffa545 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:10.846346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d97a6-d5b9-432a-a949-a0e206ffa545 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:10.858314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d97a6-d5b9-432a-a949-a0e206ffa545 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:10.859771 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d97a6-d5b9-432a-a949-a0e206ffa545 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:10.879789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 344/670] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:16:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:16:10.879789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f3ab9-5db5-4733-9c77-f70e8e7841df None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:10.880040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-bc9d97a6-d5b9-432a-a949-a0e206ffa545 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:10.880040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:10.880040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:10.880040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:10.880040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:10.880040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:10.880040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:10.880040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:10.880040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:10.880040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:10.880040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:10.880040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:10.880040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:10.880040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:10.880040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:10.880895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:10.880895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:10.880895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:10.880895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:10.880895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:10.880895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:10.880895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:10.880895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:10.880895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:10.880895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:10.880895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:10.880895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:10.880895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:10.880895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:10.880895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:10.880895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:10.880895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:10.881872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:10.881872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:10.881872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:10.881872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:10.881872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 327/671] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:10.881872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d97a6-d5b9-432a-a949-a0e206ffa545 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:10.886227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:10.917767 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:10.992301 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d97a6-d5b9-432a-a949-a0e206ffa545 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:10.992857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f3ab9-5db5-4733-9c77-f70e8e7841df None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:11.033769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2edef5-2a98-450f-9c55-41d361bad62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:11.034717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2edef5-2a98-450f-9c55-41d361bad62a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:11.035033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2edef5-2a98-450f-9c55-41d361bad62a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:11.035282 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2edef5-2a98-450f-9c55-41d361bad62a None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:11.035681 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d30b0-32d4-4956-9487-7c280f35dbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:11.036412 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2edef5-2a98-450f-9c55-41d361bad62a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:11.036600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d30b0-32d4-4956-9487-7c280f35dbe4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:11.036930 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d30b0-32d4-4956-9487-7c280f35dbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:11.037035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2edef5-2a98-450f-9c55-41d361bad62a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:11.037211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d30b0-32d4-4956-9487-7c280f35dbe4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:11.039376 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2edef5-2a98-450f-9c55-41d361bad62a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:11.044529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d30b0-32d4-4956-9487-7c280f35dbe4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:11.046011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d30b0-32d4-4956-9487-7c280f35dbe4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:11.053135 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 345/672] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:11.053865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2edef5-2a98-450f-9c55-41d361bad62a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:11.065908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f95d30b0-32d4-4956-9487-7c280f35dbe4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:11.065908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:11.065908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:11.065908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:11.065908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:11.065908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:11.065908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:11.065908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:11.065908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:11.065908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:11.065908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:11.065908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:11.065908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:11.065908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:11.065908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:11.066811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:11.066811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:11.066811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:11.066811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:11.066811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:11.066811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:11.066811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:11.066811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:11.066811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:11.066811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:11.066811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:11.066811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:11.066811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:11.066811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:11.066811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:11.066811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:11.066811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:11.067716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:11.067716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:11.067716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:11.067716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:11.067716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:11.067716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:11.067716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:11.067716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:11.067716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 328/673] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:11.068205 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d30b0-32d4-4956-9487-7c280f35dbe4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:11.143978 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:11.171289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:11.182560 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2edef5-2a98-450f-9c55-41d361bad62a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:11.190087 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d30b0-32d4-4956-9487-7c280f35dbe4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:11.223925 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b505a7-7faa-482d-ba86-3328dad36afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:11.225115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b505a7-7faa-482d-ba86-3328dad36afb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:11.225627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b505a7-7faa-482d-ba86-3328dad36afb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:11.226143 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b505a7-7faa-482d-ba86-3328dad36afb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:11.227490 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a96fe8-d0b3-4ce8-b6b3-45b5a473e7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:11.228153 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a96fe8-d0b3-4ce8-b6b3-45b5a473e7be None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:11.228463 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a96fe8-d0b3-4ce8-b6b3-45b5a473e7be None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:11.228768 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a96fe8-d0b3-4ce8-b6b3-45b5a473e7be None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:11.229495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a96fe8-d0b3-4ce8-b6b3-45b5a473e7be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:11.230061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a96fe8-d0b3-4ce8-b6b3-45b5a473e7be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:11.231443 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a96fe8-d0b3-4ce8-b6b3-45b5a473e7be None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:11.239199 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b505a7-7faa-482d-ba86-3328dad36afb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:11.240967 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b505a7-7faa-482d-ba86-3328dad36afb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:11.244632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 329/674] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:11.244632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a96fe8-d0b3-4ce8-b6b3-45b5a473e7be None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:11.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-44b505a7-7faa-482d-ba86-3328dad36afb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:11.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:11.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:11.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:11.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:11.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:11.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:11.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:11.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:11.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:11.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:11.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:11.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:11.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:11.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:11.263819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:11.263819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:11.263819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:11.263819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:11.263819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:11.263819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:11.263819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:11.263819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:11.263819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:11.263819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:11.263819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:11.263819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:11.263819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:11.263819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:11.263819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:11.263819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:11.263819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:11.265305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:11.265305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:11.265305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:11.265305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:11.265305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 346/675] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:11.265305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b505a7-7faa-482d-ba86-3328dad36afb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:11.357238 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a96fe8-d0b3-4ce8-b6b3-45b5a473e7be None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:11.374474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b505a7-7faa-482d-ba86-3328dad36afb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:11.382764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:11.394393 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660a1fc-279b-4b29-82e6-10ff302cec8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:11.395048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660a1fc-279b-4b29-82e6-10ff302cec8d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:11.395314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660a1fc-279b-4b29-82e6-10ff302cec8d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:11.395598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660a1fc-279b-4b29-82e6-10ff302cec8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:11.396572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660a1fc-279b-4b29-82e6-10ff302cec8d None admin] RBAC: Adding query filter params (role.id=f57b3e871ce142a090d6aa7dea9ac7d5, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=20e29c37dc924514a882f936f5936ae1) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:11.396893 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660a1fc-279b-4b29-82e6-10ff302cec8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:11.398447 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660a1fc-279b-4b29-82e6-10ff302cec8d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:11.412775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 330/676] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:16:11 2022] GET /identity/v3/role_assignments?user.id=20e29c37dc924514a882f936f5936ae1&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=f57b3e871ce142a090d6aa7dea9ac7d5 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:11.412775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660a1fc-279b-4b29-82e6-10ff302cec8d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:11.418939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9c708-c2e9-4bef-9d41-57baffaaae36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:11.419684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9c708-c2e9-4bef-9d41-57baffaaae36 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:11.419969 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9c708-c2e9-4bef-9d41-57baffaaae36 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:11.420284 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9c708-c2e9-4bef-9d41-57baffaaae36 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:11.421194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9c708-c2e9-4bef-9d41-57baffaaae36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:11.421508 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9c708-c2e9-4bef-9d41-57baffaaae36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:11.423145 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9c708-c2e9-4bef-9d41-57baffaaae36 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:11.439014 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 347/677] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:11.439961 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9c708-c2e9-4bef-9d41-57baffaaae36 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:11.450715 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:11.559401 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660a1fc-279b-4b29-82e6-10ff302cec8d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:11.571343 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9c708-c2e9-4bef-9d41-57baffaaae36 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:11.598783 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6954cf-203b-467c-b736-07facc234126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:11.599579 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6954cf-203b-467c-b736-07facc234126 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:11.599862 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6954cf-203b-467c-b736-07facc234126 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:11.600128 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6954cf-203b-467c-b736-07facc234126 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:11.609014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6954cf-203b-467c-b736-07facc234126 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:11.610609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3e60c-8ce8-4b35-8409-400ee5fbc527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:11.610753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6954cf-203b-467c-b736-07facc234126 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:11.610834 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3e60c-8ce8-4b35-8409-400ee5fbc527 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:11.611094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3e60c-8ce8-4b35-8409-400ee5fbc527 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:11.611325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3e60c-8ce8-4b35-8409-400ee5fbc527 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:11.613126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3e60c-8ce8-4b35-8409-400ee5fbc527 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:11.613126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3e60c-8ce8-4b35-8409-400ee5fbc527 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:11.613689 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3e60c-8ce8-4b35-8409-400ee5fbc527 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:11.626349 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 348/678] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:11.627170 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3e60c-8ce8-4b35-8409-400ee5fbc527 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:11.627364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7f6954cf-203b-467c-b736-07facc234126 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:16:11.627364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:11.627364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:11.627364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:11.627364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:11.627364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:11.627364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:11.627364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:11.627364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:11.627364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:11.627364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:11.627364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:11.627364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:11.627364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:11.627364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:11.627364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:11.628288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:11.628288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:11.628288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:11.628288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:11.628288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:11.628288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:11.628288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:11.628288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:11.628288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:11.628288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:11.628288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:11.628288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:11.628288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:11.628288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:11.628288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:11.628288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:11.628288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:11.628288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:11.629310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:11.629310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:11.629310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:11.629310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:11.629310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:11.629310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:11.629310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:11.629310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:11.629310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:16:11.629310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:11.629310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 331/679] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:16:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:11.630388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708de10-adfe-41f3-ad8f-085d126f27ce None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:11.630550 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708de10-adfe-41f3-ad8f-085d126f27ce None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:11.630828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708de10-adfe-41f3-ad8f-085d126f27ce None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:11.637132 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 332/680] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:11.637964 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708de10-adfe-41f3-ad8f-085d126f27ce None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:11.691224 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:11.703027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:11.742209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3e60c-8ce8-4b35-8409-400ee5fbc527 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:11.743195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708de10-adfe-41f3-ad8f-085d126f27ce None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:11.777421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ba708-be0e-461a-a594-0ce9746b3d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:11.777744 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14c870-985d-4db8-b8da-9a7c784a522d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:11.778077 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ba708-be0e-461a-a594-0ce9746b3d00 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:11.778359 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ba708-be0e-461a-a594-0ce9746b3d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:11.778473 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14c870-985d-4db8-b8da-9a7c784a522d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:11.778631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ba708-be0e-461a-a594-0ce9746b3d00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:11.778731 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14c870-985d-4db8-b8da-9a7c784a522d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:11.778984 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14c870-985d-4db8-b8da-9a7c784a522d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:11.779517 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ba708-be0e-461a-a594-0ce9746b3d00 None admin] RBAC: Adding query filter params (role.id=f57b3e871ce142a090d6aa7dea9ac7d5, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=29048c2021e340b081ae44a390583e7a) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:11.779769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ba708-be0e-461a-a594-0ce9746b3d00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:11.781366 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ba708-be0e-461a-a594-0ce9746b3d00 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:11.785887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14c870-985d-4db8-b8da-9a7c784a522d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:11.787321 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14c870-985d-4db8-b8da-9a7c784a522d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:11.796908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 349/681] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:16:11 2022] GET /identity/v3/role_assignments?user.id=29048c2021e340b081ae44a390583e7a&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=f57b3e871ce142a090d6aa7dea9ac7d5 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:11.796908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ba708-be0e-461a-a594-0ce9746b3d00 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:11.811776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-5c14c870-985d-4db8-b8da-9a7c784a522d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:11.811776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:11.811776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:11.811776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:11.811776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:11.811776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:11.811776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:11.811776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:11.811776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:11.811776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:11.811776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:11.811776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:11.811776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:11.811776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:11.811776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:11.812779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:11.812779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:11.812779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:11.812779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:11.812779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:11.812779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:11.812779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:11.812779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:11.812779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:11.812779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:11.812779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:11.812779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:11.812779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:11.812779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:11.812779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:11.812779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:11.812779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:11.813792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:11.813792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:11.813792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:11.813792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:11.813792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:11.813792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:11.813792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:11.813792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:11.813792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 333/682] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:11.814802 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a6410-7edb-43af-832b-a307f7fa3da0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:11.814982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a6410-7edb-43af-832b-a307f7fa3da0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:11.815258 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a6410-7edb-43af-832b-a307f7fa3da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:11.834007 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4c6a6410-7edb-43af-832b-a307f7fa3da0 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:11.835858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4c6a6410-7edb-43af-832b-a307f7fa3da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:11.920719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 334/683] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:16:11.920952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a6410-7edb-43af-832b-a307f7fa3da0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:11.937784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ba708-be0e-461a-a594-0ce9746b3d00 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:11.972193 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:11.980536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda69e02-6946-425f-9ae6-f66bf8bd5532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:11.981220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda69e02-6946-425f-9ae6-f66bf8bd5532 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:11.981497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda69e02-6946-425f-9ae6-f66bf8bd5532 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:11.982264 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda69e02-6946-425f-9ae6-f66bf8bd5532 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:11.982371 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:11.982631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda69e02-6946-425f-9ae6-f66bf8bd5532 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:11.982946 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda69e02-6946-425f-9ae6-f66bf8bd5532 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:11.984395 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda69e02-6946-425f-9ae6-f66bf8bd5532 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:12.004481 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 350/684] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Jul 18 10:16:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:12.005378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda69e02-6946-425f-9ae6-f66bf8bd5532 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:12.063200 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a6410-7edb-43af-832b-a307f7fa3da0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:12.100775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ed885-f26b-496b-a054-a3dd93757a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:12.101452 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ed885-f26b-496b-a054-a3dd93757a71 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:12.101735 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ed885-f26b-496b-a054-a3dd93757a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:12.102011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ed885-f26b-496b-a054-a3dd93757a71 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:12.102801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ed885-f26b-496b-a054-a3dd93757a71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:12.103111 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ed885-f26b-496b-a054-a3dd93757a71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:12.104743 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ed885-f26b-496b-a054-a3dd93757a71 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:12.117861 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 335/685] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:12.118708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ed885-f26b-496b-a054-a3dd93757a71 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:12.129061 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda69e02-6946-425f-9ae6-f66bf8bd5532 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:12.165913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefd45f-0701-47a0-bfd8-06eaf843621c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:12.166848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefd45f-0701-47a0-bfd8-06eaf843621c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:12.167228 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefd45f-0701-47a0-bfd8-06eaf843621c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:12.167624 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefd45f-0701-47a0-bfd8-06eaf843621c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:12.178446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefd45f-0701-47a0-bfd8-06eaf843621c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:12.180487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefd45f-0701-47a0-bfd8-06eaf843621c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:12.199364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-6aefd45f-0701-47a0-bfd8-06eaf843621c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:12.199364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:12.199364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:12.199364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:12.199364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:12.199364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:12.199364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:12.199364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:12.199364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:12.199364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:12.199364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:12.199364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:12.199364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:12.199364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:12.199364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:12.200527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:12.200527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:12.200527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:12.200527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:12.200527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:12.200527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:12.200527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:12.200527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:12.200527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:12.200527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:12.200527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:12.200527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:12.200527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:12.200527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:12.200527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:12.200527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:12.200527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:12.201442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:12.201442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:12.201442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:12.201442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:12.201925 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 351/686] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:12.202855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefd45f-0701-47a0-bfd8-06eaf843621c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:12.232531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:12.237734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ed885-f26b-496b-a054-a3dd93757a71 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:12.264627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:12.278434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0219b66-acd2-4c0c-a001-89dcef7f9dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:12.279517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0219b66-acd2-4c0c-a001-89dcef7f9dff None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:12.279896 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0219b66-acd2-4c0c-a001-89dcef7f9dff None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:12.280269 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0219b66-acd2-4c0c-a001-89dcef7f9dff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:12.344838 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefd45f-0701-47a0-bfd8-06eaf843621c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:12.353318 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0219b66-acd2-4c0c-a001-89dcef7f9dff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:12.355126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0219b66-acd2-4c0c-a001-89dcef7f9dff None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:12.387219 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc949e-cd43-4bc2-8e6c-8b6e2179b2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:12.387895 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc949e-cd43-4bc2-8e6c-8b6e2179b2ee None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:12.388179 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc949e-cd43-4bc2-8e6c-8b6e2179b2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:12.388455 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc949e-cd43-4bc2-8e6c-8b6e2179b2ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:12.389331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc949e-cd43-4bc2-8e6c-8b6e2179b2ee None admin] RBAC: Adding query filter params (role.id=f57b3e871ce142a090d6aa7dea9ac7d5, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=10d29b7934374f34923587eda9be0ed0) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:12.389641 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc949e-cd43-4bc2-8e6c-8b6e2179b2ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:12.391074 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc949e-cd43-4bc2-8e6c-8b6e2179b2ee None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:12.397454 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 352/687] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:16:12 2022] GET /identity/v3/role_assignments?user.id=10d29b7934374f34923587eda9be0ed0&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=f57b3e871ce142a090d6aa7dea9ac7d5 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:12.398265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc949e-cd43-4bc2-8e6c-8b6e2179b2ee None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:12.560682 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc949e-cd43-4bc2-8e6c-8b6e2179b2ee None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:12.564301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:12.571226 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:12.588094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 336/688] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:16:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:16:12.593472 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0219b66-acd2-4c0c-a001-89dcef7f9dff None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:12.611308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80170357-3d0a-41de-8873-97b468790f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:12.611717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80170357-3d0a-41de-8873-97b468790f78 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:12.612000 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80170357-3d0a-41de-8873-97b468790f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:12.612276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80170357-3d0a-41de-8873-97b468790f78 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:12.613055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80170357-3d0a-41de-8873-97b468790f78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:12.613361 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80170357-3d0a-41de-8873-97b468790f78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:12.614755 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80170357-3d0a-41de-8873-97b468790f78 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:12.629782 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 353/689] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:12.637404 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80170357-3d0a-41de-8873-97b468790f78 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:12.733204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0219b66-acd2-4c0c-a001-89dcef7f9dff None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:12.774414 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80170357-3d0a-41de-8873-97b468790f78 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:12.779584 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9f94e-160b-46c0-8062-e72b6bc0ad01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:12.780604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9f94e-160b-46c0-8062-e72b6bc0ad01 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:12.780986 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9f94e-160b-46c0-8062-e72b6bc0ad01 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:12.781356 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9f94e-160b-46c0-8062-e72b6bc0ad01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:12.793688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9f94e-160b-46c0-8062-e72b6bc0ad01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:12.795037 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9f94e-160b-46c0-8062-e72b6bc0ad01 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:12.814913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f8f9f94e-160b-46c0-8062-e72b6bc0ad01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:12.814913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:12.814913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:12.814913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:12.814913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:12.814913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:12.814913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:12.814913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:12.814913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:12.814913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:12.814913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:12.814913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:12.814913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:12.814913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:12.814913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:12.815987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:12.815987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:12.815987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:12.815987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:12.815987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:12.815987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:12.815987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:12.815987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:12.815987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:12.815987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:12.815987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:12.815987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:12.815987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:12.815987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:12.815987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:12.815987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:12.815987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:12.816932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:12.816932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:12.816932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:12.816932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:12.817151 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 337/690] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:12.818342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74182ffc-f918-4e55-867c-e5abb8a88cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:12.819239 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74182ffc-f918-4e55-867c-e5abb8a88cc7 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:12.819679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74182ffc-f918-4e55-867c-e5abb8a88cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:12.820122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74182ffc-f918-4e55-867c-e5abb8a88cc7 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:12.821087 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74182ffc-f918-4e55-867c-e5abb8a88cc7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:12.822917 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9f94e-160b-46c0-8062-e72b6bc0ad01 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:12.823174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74182ffc-f918-4e55-867c-e5abb8a88cc7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:12.857510 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 354/691] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 18 10:16:12 2022] POST /identity/v3/users => generated 254 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:16:12.880079 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:12.957742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9f94e-160b-46c0-8062-e72b6bc0ad01 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:13.002685 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a293bd6-5dfa-4fe1-b177-d2b3587a54b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:13.003730 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a293bd6-5dfa-4fe1-b177-d2b3587a54b1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:13.004199 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a293bd6-5dfa-4fe1-b177-d2b3587a54b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:13.004685 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a293bd6-5dfa-4fe1-b177-d2b3587a54b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:13.005641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a293bd6-5dfa-4fe1-b177-d2b3587a54b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:13.006169 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a293bd6-5dfa-4fe1-b177-d2b3587a54b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:13.007835 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a293bd6-5dfa-4fe1-b177-d2b3587a54b1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:13.026816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 338/692] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:13.032280 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74182ffc-f918-4e55-867c-e5abb8a88cc7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:13.078302 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:13.169800 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74182ffc-f918-4e55-867c-e5abb8a88cc7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:13.211400 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c80c3-03cf-413c-9eeb-7fcaa5a06337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:13.212233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c80c3-03cf-413c-9eeb-7fcaa5a06337 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:13.212536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c80c3-03cf-413c-9eeb-7fcaa5a06337 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:13.212823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c80c3-03cf-413c-9eeb-7fcaa5a06337 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:13.222004 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c80c3-03cf-413c-9eeb-7fcaa5a06337 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:13.223493 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c80c3-03cf-413c-9eeb-7fcaa5a06337 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:13.238941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-128c80c3-03cf-413c-9eeb-7fcaa5a06337 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:13.238941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:13.238941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:13.238941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:13.238941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:13.238941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:13.238941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:13.238941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:13.238941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:13.238941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:13.238941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:13.238941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:13.238941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:13.238941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:13.238941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:13.239966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:13.239966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:13.239966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:13.239966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:13.239966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:13.239966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:13.239966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:13.239966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:13.239966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:13.239966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:13.239966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:13.239966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:13.239966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:13.239966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:13.239966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:13.239966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:13.239966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:13.240900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:13.240900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:13.240900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:13.240900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:13.240900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:13.240900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:13.240900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:13.240900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:13.240900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 355/693] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:13.246010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a293bd6-5dfa-4fe1-b177-d2b3587a54b1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:13.356950 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a293bd6-5dfa-4fe1-b177-d2b3587a54b1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:13.368950 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:13.392573 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97b68d-325b-4758-9bf0-653c7609cafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:13.393239 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97b68d-325b-4758-9bf0-653c7609cafb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:13.393577 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97b68d-325b-4758-9bf0-653c7609cafb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:13.393911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97b68d-325b-4758-9bf0-653c7609cafb None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:13.394818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97b68d-325b-4758-9bf0-653c7609cafb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:13.395070 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97b68d-325b-4758-9bf0-653c7609cafb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:13.396499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97b68d-325b-4758-9bf0-653c7609cafb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:13.411119 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 339/694] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:13.417516 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c80c3-03cf-413c-9eeb-7fcaa5a06337 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:13.513145 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:13.539353 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c80c3-03cf-413c-9eeb-7fcaa5a06337 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:13.586930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bcb6d-3096-4893-888c-f24747e6fbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:13.587595 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bcb6d-3096-4893-888c-f24747e6fbef None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:13.587878 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bcb6d-3096-4893-888c-f24747e6fbef None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:13.588155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bcb6d-3096-4893-888c-f24747e6fbef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:13.589029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bcb6d-3096-4893-888c-f24747e6fbef None admin] RBAC: Adding query filter params (role.id=f57b3e871ce142a090d6aa7dea9ac7d5, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=fd6168873f2846c8af73b903bba17eb8) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:13.589343 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bcb6d-3096-4893-888c-f24747e6fbef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:13.590787 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bcb6d-3096-4893-888c-f24747e6fbef None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:13.598582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 356/695] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:16:13 2022] GET /identity/v3/role_assignments?user.id=fd6168873f2846c8af73b903bba17eb8&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=f57b3e871ce142a090d6aa7dea9ac7d5 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:13.622671 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231729b4-61bb-464e-b5b6-68239aff2f4d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:13.622964 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231729b4-61bb-464e-b5b6-68239aff2f4d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:13.623245 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231729b4-61bb-464e-b5b6-68239aff2f4d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:13.624882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 340/696] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:13.638741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8f2c8-9010-46e8-9eea-adb611b5a23b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:13.639056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8f2c8-9010-46e8-9eea-adb611b5a23b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:13.639309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8f2c8-9010-46e8-9eea-adb611b5a23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:13.660904 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e5e8f2c8-9010-46e8-9eea-adb611b5a23b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:13.662139 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e5e8f2c8-9010-46e8-9eea-adb611b5a23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:13.750939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 357/697] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:16:13.796932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231729b4-61bb-464e-b5b6-68239aff2f4d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:13.930024 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231729b4-61bb-464e-b5b6-68239aff2f4d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:13.965648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d065b4-d8c6-48a0-99c0-b9a6e7f594e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:13.966456 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d065b4-d8c6-48a0-99c0-b9a6e7f594e0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:13.966738 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d065b4-d8c6-48a0-99c0-b9a6e7f594e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:13.967072 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d065b4-d8c6-48a0-99c0-b9a6e7f594e0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:13.967730 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d065b4-d8c6-48a0-99c0-b9a6e7f594e0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:13.969228 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d065b4-d8c6-48a0-99c0-b9a6e7f594e0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:13.983869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-13d065b4-d8c6-48a0-99c0-b9a6e7f594e0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:16:13.983869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:13.983869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:13.983869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:13.983869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:13.983869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:13.983869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:13.983869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:13.983869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:16:13.983869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:13.983869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:13.983869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:13.983869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:13.983869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:13.983869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:13.983869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:13.984962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:13.984962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:13.984962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:13.984962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:13.984962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:13.984962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:13.984962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:13.984962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:13.984962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:13.984962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:13.984962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:13.984962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:13.984962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:13.984962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:13.984962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:13.984962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:13.986217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:13.986217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:13.986217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:13.986217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:13.986217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:13.986217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:13.986217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:13.986217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:13.986217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:13.986217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:13.986217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:13.986217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:13.986217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:13.986217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:16:13.986217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:13.986217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 341/698] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Jul 18 10:16:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:13.991162 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8f2c8-9010-46e8-9eea-adb611b5a23b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:14.075212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:14.103719 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8f2c8-9010-46e8-9eea-adb611b5a23b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:14.142111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b23dc-1be5-4351-b275-ceb4c56f13e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:14.142794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b23dc-1be5-4351-b275-ceb4c56f13e2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:14.143076 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b23dc-1be5-4351-b275-ceb4c56f13e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:14.143350 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b23dc-1be5-4351-b275-ceb4c56f13e2 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:14.144081 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b23dc-1be5-4351-b275-ceb4c56f13e2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:14.144486 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b23dc-1be5-4351-b275-ceb4c56f13e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:14.146091 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b23dc-1be5-4351-b275-ceb4c56f13e2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:14.151744 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 358/699] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Jul 18 10:16:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:14.160615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d065b4-d8c6-48a0-99c0-b9a6e7f594e0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:14.198360 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:14.266382 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d065b4-d8c6-48a0-99c0-b9a6e7f594e0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:14.296126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b574bd-3853-4a25-8f3c-6fbabc38d387 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:14.296413 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b574bd-3853-4a25-8f3c-6fbabc38d387 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:14.296738 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b574bd-3853-4a25-8f3c-6fbabc38d387 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:14.298760 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 359/700] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:16:14.304184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095182a-a5c5-49b4-9073-2e5ff7b4f05f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:14.304478 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095182a-a5c5-49b4-9073-2e5ff7b4f05f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:14.304897 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095182a-a5c5-49b4-9073-2e5ff7b4f05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:14.307679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bc2b7-f534-416a-9c3b-f80e0b482745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:14.308347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bc2b7-f534-416a-9c3b-f80e0b482745 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:14.308665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bc2b7-f534-416a-9c3b-f80e0b482745 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:14.308949 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bc2b7-f534-416a-9c3b-f80e0b482745 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:14.309672 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bc2b7-f534-416a-9c3b-f80e0b482745 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:14.309985 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bc2b7-f534-416a-9c3b-f80e0b482745 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:14.311364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bc2b7-f534-416a-9c3b-f80e0b482745 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:14.322932 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1095182a-a5c5-49b4-9073-2e5ff7b4f05f None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:14.324413 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1095182a-a5c5-49b4-9073-2e5ff7b4f05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:14.326823 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 342/701] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Jul 18 10:16:14 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:14.442222 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 360/702] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:16:14.477703 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bc2b7-f534-416a-9c3b-f80e0b482745 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:14.622647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:14.627324 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bc2b7-f534-416a-9c3b-f80e0b482745 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:14.663983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b169917-7bb4-4345-8eeb-34d0d9b858a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:14.664822 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b169917-7bb4-4345-8eeb-34d0d9b858a0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:14.665110 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b169917-7bb4-4345-8eeb-34d0d9b858a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:14.665390 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b169917-7bb4-4345-8eeb-34d0d9b858a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:14.681411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b169917-7bb4-4345-8eeb-34d0d9b858a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:14.689080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9b169917-7bb4-4345-8eeb-34d0d9b858a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:16:14.689080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:14.689080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:14.689080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:14.689080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:14.689080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:14.689080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:14.689080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:14.689080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:14.689080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:14.689080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:14.689080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:14.689080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:14.689080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:14.689080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:14.689080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:14.689993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:14.689993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:14.689993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:14.689993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:14.689993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:14.689993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:14.689993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:14.689993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:14.689993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:14.689993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:14.689993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:14.689993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:14.689993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:14.689993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:14.689993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:14.689993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:14.690864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:14.690864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:14.690864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:14.690864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:14.690864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:14.690864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:14.690864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:14.690864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:14.690864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:14.690864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:14.690864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:16:14.690864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:14.690864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 343/703] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:16:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:14.700463 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095182a-a5c5-49b4-9073-2e5ff7b4f05f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:14.806401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:14.847961 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095182a-a5c5-49b4-9073-2e5ff7b4f05f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:14.897560 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad1fdf-01c2-4cc9-b86f-8bf8b6e51a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:14.898612 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad1fdf-01c2-4cc9-b86f-8bf8b6e51a52 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:14.899124 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad1fdf-01c2-4cc9-b86f-8bf8b6e51a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:14.899595 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad1fdf-01c2-4cc9-b86f-8bf8b6e51a52 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:14.900572 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad1fdf-01c2-4cc9-b86f-8bf8b6e51a52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:14.901079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad1fdf-01c2-4cc9-b86f-8bf8b6e51a52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:14.902665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad1fdf-01c2-4cc9-b86f-8bf8b6e51a52 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:14.917064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 361/704] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:16:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:14.922838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b169917-7bb4-4345-8eeb-34d0d9b858a0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:15.065356 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b169917-7bb4-4345-8eeb-34d0d9b858a0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:15.117696 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a74e0-102b-4712-b179-bfe7a856f8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:15.118499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a74e0-102b-4712-b179-bfe7a856f8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:15.118778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a74e0-102b-4712-b179-bfe7a856f8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:15.119068 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a74e0-102b-4712-b179-bfe7a856f8d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:15.140517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a74e0-102b-4712-b179-bfe7a856f8d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:15.142013 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a74e0-102b-4712-b179-bfe7a856f8d0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:15.163616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-845a74e0-102b-4712-b179-bfe7a856f8d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:15.163616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:15.163616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:15.163616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:15.163616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:15.163616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:15.163616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:15.163616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:15.163616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:15.163616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:15.163616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:15.163616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:15.163616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:15.163616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:15.163616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:15.164488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:15.164488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:15.164488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:15.164488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:15.164488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:15.164488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:15.164488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:15.164488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:15.164488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:15.164488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:15.164488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:15.164488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:15.164488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:15.164488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:15.164488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:15.164488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:15.164488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:15.165420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:15.165420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:15.165420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:15.165420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:15.165420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 344/705] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:15.171942 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad1fdf-01c2-4cc9-b86f-8bf8b6e51a52 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:15.322045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad1fdf-01c2-4cc9-b86f-8bf8b6e51a52 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:15.358668 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a3f3c-90f8-43ca-964c-f7bd01f68307 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:15.359608 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a3f3c-90f8-43ca-964c-f7bd01f68307 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:15.360103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a3f3c-90f8-43ca-964c-f7bd01f68307 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:15.361607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:15.362115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 345/706] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:16:15.368136 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72de71-85d1-4ad3-94a2-4b87a03648a4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:15.368824 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72de71-85d1-4ad3-94a2-4b87a03648a4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:15.369414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72de71-85d1-4ad3-94a2-4b87a03648a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:15.371119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820397be-6631-48e4-bac3-0fbb387a154b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:15.371807 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820397be-6631-48e4-bac3-0fbb387a154b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:15.372085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820397be-6631-48e4-bac3-0fbb387a154b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:15.372367 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820397be-6631-48e4-bac3-0fbb387a154b None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:15.373134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820397be-6631-48e4-bac3-0fbb387a154b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:15.373446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820397be-6631-48e4-bac3-0fbb387a154b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:15.374826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820397be-6631-48e4-bac3-0fbb387a154b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:15.388934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-dd72de71-85d1-4ad3-94a2-4b87a03648a4 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:15.390418 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dd72de71-85d1-4ad3-94a2-4b87a03648a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:15.393521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:15.398595 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 362/707] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:15.414042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820397be-6631-48e4-bac3-0fbb387a154b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:15.483134 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 346/708] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:16:15.493010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf83119-3101-4078-93e4-1a0858981ca5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:15.493621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf83119-3101-4078-93e4-1a0858981ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:15.494098 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf83119-3101-4078-93e4-1a0858981ca5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:15.496054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 347/709] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:15 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:15.514016 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63d072-68dc-4f2e-98f0-30bbeaefc2b8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:15.514633 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63d072-68dc-4f2e-98f0-30bbeaefc2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:15.515104 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63d072-68dc-4f2e-98f0-30bbeaefc2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:15.532374 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-db63d072-68dc-4f2e-98f0-30bbeaefc2b8 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:15.533872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-db63d072-68dc-4f2e-98f0-30bbeaefc2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:15.546969 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820397be-6631-48e4-bac3-0fbb387a154b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:15.586503 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02818a-ba36-4aef-923f-ced3786bf224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:15.587888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02818a-ba36-4aef-923f-ced3786bf224 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:15.588545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02818a-ba36-4aef-923f-ced3786bf224 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:15.589021 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02818a-ba36-4aef-923f-ced3786bf224 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:15.597143 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a02818a-ba36-4aef-923f-ced3786bf224 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:15.598848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a02818a-ba36-4aef-923f-ced3786bf224 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:15.608078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 348/710] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 10:16:15.609259 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63d072-68dc-4f2e-98f0-30bbeaefc2b8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:15.623105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8a02818a-ba36-4aef-923f-ced3786bf224 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:16:15.623105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:15.623105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:15.623105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:15.623105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:15.623105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:15.623105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:15.623105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:15.623105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:15.623105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:15.623105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:15.623105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:15.623105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:15.623105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:15.623105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:15.623105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:15.624326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:15.624326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:15.624326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:15.624326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:15.624326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:15.624326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:15.624326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:15.624326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:15.624326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:15.624326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:15.624326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:15.624326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:15.624326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:15.624326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:15.624326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:15.624326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:15.624326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:15.624326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:15.625570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:15.625570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:15.625570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:15.625570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:15.625570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:15.625570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:15.625570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:15.625570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:15.625570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:16:15.625570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:15.627339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 363/711] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jul 18 10:16:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:15.632700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02818a-ba36-4aef-923f-ced3786bf224 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:15.686769 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:15.698512 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:15.722982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63d072-68dc-4f2e-98f0-30bbeaefc2b8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:15.750346 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02818a-ba36-4aef-923f-ced3786bf224 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:15.760991 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513cc20-b8eb-4219-bc90-285265a32221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:15.762137 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513cc20-b8eb-4219-bc90-285265a32221 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:15.762638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513cc20-b8eb-4219-bc90-285265a32221 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:15.763116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513cc20-b8eb-4219-bc90-285265a32221 None admin] PATH_INFO: `/v3/services/network` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:15.764103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5513cc20-b8eb-4219-bc90-285265a32221 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:15.765704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5513cc20-b8eb-4219-bc90-285265a32221 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:15.781048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-5513cc20-b8eb-4219-bc90-285265a32221 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:16:15.781048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:15.781048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:15.781048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:15.781048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:15.781048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:15.781048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:15.781048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:15.781048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:16:15.781048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:15.781048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:15.781048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:15.781048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:15.781048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:15.781048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:15.781048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:15.781048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:15.782054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:15.782054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:15.782054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:15.782054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:15.782054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:15.782054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:15.782054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:15.782054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:15.782054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:15.782054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:15.782054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:15.782054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:15.782054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:15.782054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:15.782054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:15.782054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:15.782933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:15.782933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:15.782933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:15.782933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:15.782933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:15.782933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:15.782933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:15.782933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:15.782933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:15.782933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:15.782933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:15.782933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:15.782933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:16:15.782933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:15.786354 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 349/712] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:15 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:15.787438 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513cc20-b8eb-4219-bc90-285265a32221 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:15.792882 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893de0f-672c-452e-9a68-e4396c4e302c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:15.793561 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893de0f-672c-452e-9a68-e4396c4e302c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:15.793844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893de0f-672c-452e-9a68-e4396c4e302c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:15.794116 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893de0f-672c-452e-9a68-e4396c4e302c None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:15.794968 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893de0f-672c-452e-9a68-e4396c4e302c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:15.795277 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893de0f-672c-452e-9a68-e4396c4e302c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:15.796829 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893de0f-672c-452e-9a68-e4396c4e302c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:15.819638 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 364/713] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jul 18 10:16:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:15.820650 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893de0f-672c-452e-9a68-e4396c4e302c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:15.899844 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513cc20-b8eb-4219-bc90-285265a32221 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:15.925899 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893de0f-672c-452e-9a68-e4396c4e302c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:15.935432 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:15.939440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d30a0-6737-4bb0-8948-adfccc616726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:15.940625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d30a0-6737-4bb0-8948-adfccc616726 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:15.941156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d30a0-6737-4bb0-8948-adfccc616726 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:15.942352 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d30a0-6737-4bb0-8948-adfccc616726 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:15.944916 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:15.956937 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d30a0-6737-4bb0-8948-adfccc616726 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:15.962479 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0b56a-65ed-45cf-aaab-8592383f288d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:15.963142 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0b56a-65ed-45cf-aaab-8592383f288d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:15.963426 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0b56a-65ed-45cf-aaab-8592383f288d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:15.963703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0b56a-65ed-45cf-aaab-8592383f288d None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:15.964409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0b56a-65ed-45cf-aaab-8592383f288d None admin] RBAC: Adding query filter params (name=network) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:15.964741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0b56a-65ed-45cf-aaab-8592383f288d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:15.966104 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0b56a-65ed-45cf-aaab-8592383f288d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:15.967838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9b1d30a0-6737-4bb0-8948-adfccc616726 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:15.967838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:15.967838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:15.967838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:15.967838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:15.967838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:15.967838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:15.967838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:15.967838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:15.967838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:15.967838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:15.967838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:15.967838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:15.967838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:15.967838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:15.967838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:15.968845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:15.968845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:15.968845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:15.968845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:15.968845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:15.968845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:15.968845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:15.968845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:15.968845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:15.968845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:15.968845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:15.968845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:15.968845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:15.968845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:15.968845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:15.968845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:15.969809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:15.969809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:15.969809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:15.969809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:15.969809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:15.969809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:15.969809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:15.969809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:15.969809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:15.969809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:15.969809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:15.969809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:15.972649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 350/715] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:16:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:15.973756 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d30a0-6737-4bb0-8948-adfccc616726 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:15.976821 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 365/715] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 18 10:16:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:15.977774 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0b56a-65ed-45cf-aaab-8592383f288d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:16.084559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0b56a-65ed-45cf-aaab-8592383f288d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:16.090889 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d30a0-6737-4bb0-8948-adfccc616726 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:16.120439 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b89b4-9d8b-4da5-bc5b-d9fde7052b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:16.121116 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b89b4-9d8b-4da5-bc5b-d9fde7052b05 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:16.121397 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b89b4-9d8b-4da5-bc5b-d9fde7052b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:16.121669 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b89b4-9d8b-4da5-bc5b-d9fde7052b05 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:16.122380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b89b4-9d8b-4da5-bc5b-d9fde7052b05 None admin] RBAC: Adding query filter params (type=network) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:16.122671 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b89b4-9d8b-4da5-bc5b-d9fde7052b05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:16.124028 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b89b4-9d8b-4da5-bc5b-d9fde7052b05 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:16.130176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005ca6b-f9ef-4a61-8256-d7ea15704c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:16.131161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005ca6b-f9ef-4a61-8256-d7ea15704c1c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:16.132183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005ca6b-f9ef-4a61-8256-d7ea15704c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:16.132728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005ca6b-f9ef-4a61-8256-d7ea15704c1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:16.136878 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 366/716] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 18 10:16:15 2022] GET /identity/v3/services?type=network => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:16.138015 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b89b4-9d8b-4da5-bc5b-d9fde7052b05 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:16.204659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:16.207655 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:16.220481 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005ca6b-f9ef-4a61-8256-d7ea15704c1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:16.222420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005ca6b-f9ef-4a61-8256-d7ea15704c1c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:16.326018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b89b4-9d8b-4da5-bc5b-d9fde7052b05 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:16.336975 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 351/717] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:16:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:16:16.342255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005ca6b-f9ef-4a61-8256-d7ea15704c1c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:16.383768 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6c3c8-f9b4-452b-8acf-51009cbabcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:16.384944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6c3c8-f9b4-452b-8acf-51009cbabcf0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:16.385437 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6c3c8-f9b4-452b-8acf-51009cbabcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:16.387188 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6c3c8-f9b4-452b-8acf-51009cbabcf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:16.388285 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6c3c8-f9b4-452b-8acf-51009cbabcf0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:16.392391 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6c3c8-f9b4-452b-8acf-51009cbabcf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:16.394132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6c3c8-f9b4-452b-8acf-51009cbabcf0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:16.413984 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 367/718] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:16:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:16.416443 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6c3c8-f9b4-452b-8acf-51009cbabcf0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:16.496804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:16.505053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005ca6b-f9ef-4a61-8256-d7ea15704c1c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:16.533412 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:16.538471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6c3c8-f9b4-452b-8acf-51009cbabcf0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:16.550543 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f8989-2b34-4e8a-89e6-4566efa7d183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:16.551921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f8989-2b34-4e8a-89e6-4566efa7d183 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:16.552431 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f8989-2b34-4e8a-89e6-4566efa7d183 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:16.553204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f8989-2b34-4e8a-89e6-4566efa7d183 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:16.568577 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f8989-2b34-4e8a-89e6-4566efa7d183 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:16.571710 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f8989-2b34-4e8a-89e6-4566efa7d183 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:16.576257 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14b4ca-53d3-4f0d-b688-a3811a8b718f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:16.577158 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14b4ca-53d3-4f0d-b688-a3811a8b718f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:16.577448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14b4ca-53d3-4f0d-b688-a3811a8b718f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:16.577731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14b4ca-53d3-4f0d-b688-a3811a8b718f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:16.594799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14b4ca-53d3-4f0d-b688-a3811a8b718f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:16.596288 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14b4ca-53d3-4f0d-b688-a3811a8b718f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:16.602850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ce8f8989-2b34-4e8a-89e6-4566efa7d183 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:16.602850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:16.602850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:16.602850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:16.602850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:16.602850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:16.602850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:16.602850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:16.602850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:16.602850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:16.602850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:16.602850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:16.602850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:16.602850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:16.602850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:16.603809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:16.603809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:16.603809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:16.603809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:16.603809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:16.603809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:16.603809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:16.603809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:16.603809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:16.603809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:16.603809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:16.603809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:16.603809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:16.603809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:16.603809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:16.603809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:16.603809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:16.604744 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:16.604744 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:16.604744 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:16.604744 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:16.608560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 352/719] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:16.610252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852218f-dbfc-4b52-aa67-62c4107fd09d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:16.610764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852218f-dbfc-4b52-aa67-62c4107fd09d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:16.611254 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852218f-dbfc-4b52-aa67-62c4107fd09d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:16.613276 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 353/720] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:16.614250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852218f-dbfc-4b52-aa67-62c4107fd09d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:16.626154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-3f14b4ca-53d3-4f0d-b688-a3811a8b718f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:16.626154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:16.626154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:16.626154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:16.626154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:16.626154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:16.626154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:16.626154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:16.626154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:16.626154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:16.626154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:16.626154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:16.626154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:16.626154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:16.626154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:16.627103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:16.627103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:16.627103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:16.627103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:16.627103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:16.627103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:16.627103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:16.627103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:16.627103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:16.627103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:16.627103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:16.627103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:16.627103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:16.627103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:16.627103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:16.627103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:16.627103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:16.628641 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:16.628641 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:16.628641 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:16.628641 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:16.628641 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 368/721] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:16.629060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda9184-91d1-48c7-b620-397f57df16b2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:16.629340 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda9184-91d1-48c7-b620-397f57df16b2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:16.629696 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda9184-91d1-48c7-b620-397f57df16b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:16.647744 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5bda9184-91d1-48c7-b620-397f57df16b2 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:16.649040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5bda9184-91d1-48c7-b620-397f57df16b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:16.732724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 369/722] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:16:16.734310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda9184-91d1-48c7-b620-397f57df16b2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:16.769296 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852218f-dbfc-4b52-aa67-62c4107fd09d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:16.819146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee94cd5-f47c-4a2d-ac7e-691a568449da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:16.821021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee94cd5-f47c-4a2d-ac7e-691a568449da None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:16.822462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee94cd5-f47c-4a2d-ac7e-691a568449da None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:16.822951 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee94cd5-f47c-4a2d-ac7e-691a568449da None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:16.823914 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee94cd5-f47c-4a2d-ac7e-691a568449da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:16.824427 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee94cd5-f47c-4a2d-ac7e-691a568449da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:16.826604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee94cd5-f47c-4a2d-ac7e-691a568449da None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:16.832736 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:16.847437 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 354/723] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:16.848673 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee94cd5-f47c-4a2d-ac7e-691a568449da None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:16.859416 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda9184-91d1-48c7-b620-397f57df16b2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:16.860986 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:16.899116 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f37d76-05b2-4262-91b5-d9e8a6e92a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:16.900205 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f37d76-05b2-4262-91b5-d9e8a6e92a02 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:16.900205 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f37d76-05b2-4262-91b5-d9e8a6e92a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:16.900415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f37d76-05b2-4262-91b5-d9e8a6e92a02 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:16.901141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f37d76-05b2-4262-91b5-d9e8a6e92a02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:16.901459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f37d76-05b2-4262-91b5-d9e8a6e92a02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:16.902853 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f37d76-05b2-4262-91b5-d9e8a6e92a02 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:16.915761 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 370/724] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:16.916730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f37d76-05b2-4262-91b5-d9e8a6e92a02 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:16.977715 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee94cd5-f47c-4a2d-ac7e-691a568449da None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:17.013855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6f570-52f5-4241-af05-86041a015e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:17.014582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6f570-52f5-4241-af05-86041a015e02 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:17.014870 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6f570-52f5-4241-af05-86041a015e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:17.015150 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6f570-52f5-4241-af05-86041a015e02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:17.024194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f37d76-05b2-4262-91b5-d9e8a6e92a02 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:17.025762 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be6f570-52f5-4241-af05-86041a015e02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:17.033389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8be6f570-52f5-4241-af05-86041a015e02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:16:17.033389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:17.033389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:17.033389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:17.033389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:17.033389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:17.033389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:17.033389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:17.033389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:17.033389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:17.033389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:17.033389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:17.033389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:17.033389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:17.033389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:17.033389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:17.034363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:17.034363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:17.034363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:17.034363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:17.034363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:17.034363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:17.034363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:17.034363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:17.034363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:17.034363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:17.034363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:17.034363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:17.034363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:17.034363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:17.034363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:17.034363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:17.035307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:17.035307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:17.035307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:17.035307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:17.035307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:17.035307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:17.035307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:17.035307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:17.035307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:17.035307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:17.035307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:16:17.035307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:17.040424 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 355/725] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:16:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:17.041287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6f570-52f5-4241-af05-86041a015e02 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:17.065062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782453d8-eee6-41f6-84f3-066bbdac1e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:17.065863 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782453d8-eee6-41f6-84f3-066bbdac1e26 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:17.066144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782453d8-eee6-41f6-84f3-066bbdac1e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:17.066418 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782453d8-eee6-41f6-84f3-066bbdac1e26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:17.073910 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782453d8-eee6-41f6-84f3-066bbdac1e26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:17.075993 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782453d8-eee6-41f6-84f3-066bbdac1e26 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:17.079156 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:17.095129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-782453d8-eee6-41f6-84f3-066bbdac1e26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:17.095129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:17.095129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:17.095129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:17.095129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:17.095129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:17.095129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:17.095129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:17.095129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:17.095129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:17.095129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:17.095129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:17.095129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:17.095129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:17.095129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:17.096336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:17.096336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:17.096336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:17.096336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:17.096336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:17.096336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:17.096336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:17.096336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:17.096336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:17.096336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:17.096336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:17.096336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:17.096336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:17.096336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:17.096336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:17.096336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:17.096336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:17.097517 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:17.097517 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:17.097517 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:17.097517 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:17.097517 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:17.097517 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:17.097517 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:17.097517 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:17.105792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 371/726] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:17.105792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d17cdf-7400-4842-a93d-e9ccf5af18b4 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:17.105792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d17cdf-7400-4842-a93d-e9ccf5af18b4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:17.105792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d17cdf-7400-4842-a93d-e9ccf5af18b4 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:17.105792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 372/727] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:17.106686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d17cdf-7400-4842-a93d-e9ccf5af18b4 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:17.114240 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:17.156144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6f570-52f5-4241-af05-86041a015e02 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:17.205506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0912a8d-56b7-4d7f-ba44-9973f055121a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:17.206341 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0912a8d-56b7-4d7f-ba44-9973f055121a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:17.206630 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0912a8d-56b7-4d7f-ba44-9973f055121a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:17.206914 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0912a8d-56b7-4d7f-ba44-9973f055121a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:17.214967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0912a8d-56b7-4d7f-ba44-9973f055121a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:17.216433 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0912a8d-56b7-4d7f-ba44-9973f055121a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:17.226902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d17cdf-7400-4842-a93d-e9ccf5af18b4 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:17.231847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c0912a8d-56b7-4d7f-ba44-9973f055121a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:16:17.231847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:17.231847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:17.231847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:17.231847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:17.231847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:17.231847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:17.231847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:17.231847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:17.231847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:17.231847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:17.231847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:17.231847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:17.231847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:17.231847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:17.231847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:17.232819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:17.232819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:17.232819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:17.232819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:17.232819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:17.232819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:17.232819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:17.232819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:17.232819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:17.232819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:17.232819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:17.232819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:17.232819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:17.232819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:17.232819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:17.232819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:17.232819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:17.232819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:17.239094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:17.239094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:17.239094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:17.239094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:17.239094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:17.239094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:17.239094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:17.239094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:17.239094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:16:17.239094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:17.239094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 356/728] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 18 10:16:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:17.239094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0912a8d-56b7-4d7f-ba44-9973f055121a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:17.268041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134d88e-4c76-4abe-af3d-a16774601d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:17.268667 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134d88e-4c76-4abe-af3d-a16774601d4b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:17.268954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134d88e-4c76-4abe-af3d-a16774601d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:17.269231 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134d88e-4c76-4abe-af3d-a16774601d4b None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:17.269980 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d134d88e-4c76-4abe-af3d-a16774601d4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:17.270289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d134d88e-4c76-4abe-af3d-a16774601d4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:17.271657 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d134d88e-4c76-4abe-af3d-a16774601d4b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:17.284482 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 373/729] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:16:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:17.284482 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7562ba-40eb-4b16-9dcf-8ef27bb9c72c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:17.284482 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7562ba-40eb-4b16-9dcf-8ef27bb9c72c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:17.284482 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7562ba-40eb-4b16-9dcf-8ef27bb9c72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:17.299301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0f7562ba-40eb-4b16-9dcf-8ef27bb9c72c None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:17.301287 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0f7562ba-40eb-4b16-9dcf-8ef27bb9c72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:17.348793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0912a8d-56b7-4d7f-ba44-9973f055121a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:17.361068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:17.373591 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 374/730] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:16:17.374429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7562ba-40eb-4b16-9dcf-8ef27bb9c72c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:17.387519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:17.397114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad810f-4831-438f-a7d9-dfa8b557a4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:17.398420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad810f-4831-438f-a7d9-dfa8b557a4be None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:17.398770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad810f-4831-438f-a7d9-dfa8b557a4be None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:17.399169 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad810f-4831-438f-a7d9-dfa8b557a4be None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:17.400277 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad810f-4831-438f-a7d9-dfa8b557a4be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:17.400616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad810f-4831-438f-a7d9-dfa8b557a4be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:17.402046 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad810f-4831-438f-a7d9-dfa8b557a4be None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:17.415856 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 357/731] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:17.416495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad810f-4831-438f-a7d9-dfa8b557a4be None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:17.492276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7562ba-40eb-4b16-9dcf-8ef27bb9c72c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:17.523778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad810f-4831-438f-a7d9-dfa8b557a4be None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:17.528581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33535183-d847-4f1e-a047-2c3f97dc80a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:17.529262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33535183-d847-4f1e-a047-2c3f97dc80a8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:17.529549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33535183-d847-4f1e-a047-2c3f97dc80a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:17.529828 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33535183-d847-4f1e-a047-2c3f97dc80a8 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:17.530676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33535183-d847-4f1e-a047-2c3f97dc80a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:17.530987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33535183-d847-4f1e-a047-2c3f97dc80a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:17.532414 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33535183-d847-4f1e-a047-2c3f97dc80a8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:17.556915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 375/732] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Jul 18 10:16:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:17.556915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33535183-d847-4f1e-a047-2c3f97dc80a8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:17.560649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3605a45-1f13-430b-87e0-d908c911b048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:17.561451 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3605a45-1f13-430b-87e0-d908c911b048 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:17.561740 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3605a45-1f13-430b-87e0-d908c911b048 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:17.562019 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3605a45-1f13-430b-87e0-d908c911b048 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:17.572407 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3605a45-1f13-430b-87e0-d908c911b048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:17.573890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3605a45-1f13-430b-87e0-d908c911b048 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:17.592207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f3605a45-1f13-430b-87e0-d908c911b048 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:17.592207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:17.592207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:17.592207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:17.592207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:17.592207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:17.592207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:17.592207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:17.592207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:17.592207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:17.592207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:17.592207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:17.592207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:17.592207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:17.592207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:17.593156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:17.593156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:17.593156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:17.593156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:17.593156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:17.593156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:17.593156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:17.593156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:17.593156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:17.593156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:17.593156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:17.593156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:17.593156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:17.593156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:17.593156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:17.593156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:17.593156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:17.594042 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:17.594042 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:17.594042 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:17.594042 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:17.598088 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 358/733] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:17.598733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3605a45-1f13-430b-87e0-d908c911b048 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:17.612036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:17.642496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:17.668598 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33535183-d847-4f1e-a047-2c3f97dc80a8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:17.706010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3605a45-1f13-430b-87e0-d908c911b048 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:17.708309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc245a-f64a-464d-bc9d-d68aa9f0edb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:17.708994 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc245a-f64a-464d-bc9d-d68aa9f0edb5 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:17.709278 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc245a-f64a-464d-bc9d-d68aa9f0edb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:17.709549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc245a-f64a-464d-bc9d-d68aa9f0edb5 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:17.711184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc245a-f64a-464d-bc9d-d68aa9f0edb5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:17.712584 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc245a-f64a-464d-bc9d-d68aa9f0edb5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:17.737449 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 376/734] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 18 10:16:17 2022] POST /identity/v3/services => generated 251 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:16:17.739458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc245a-f64a-464d-bc9d-d68aa9f0edb5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:17.769263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39329473-3d5a-43d1-8559-96d6c4a4b9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:17.770667 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39329473-3d5a-43d1-8559-96d6c4a4b9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:17.771111 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39329473-3d5a-43d1-8559-96d6c4a4b9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:17.771571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39329473-3d5a-43d1-8559-96d6c4a4b9e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:17.772861 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39329473-3d5a-43d1-8559-96d6c4a4b9e2 None admin] RBAC: Adding query filter params (role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=29048c2021e340b081ae44a390583e7a) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:17.773341 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39329473-3d5a-43d1-8559-96d6c4a4b9e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:17.776229 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39329473-3d5a-43d1-8559-96d6c4a4b9e2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:17.793033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 359/735] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:16:17 2022] GET /identity/v3/role_assignments?user.id=29048c2021e340b081ae44a390583e7a&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:17.794296 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39329473-3d5a-43d1-8559-96d6c4a4b9e2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:17.892768 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc245a-f64a-464d-bc9d-d68aa9f0edb5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:17.907981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:17.941255 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6025f-54d4-49ee-8075-904b20f938e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:17.942450 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6025f-54d4-49ee-8075-904b20f938e2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:17.942837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6025f-54d4-49ee-8075-904b20f938e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:17.943526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6025f-54d4-49ee-8075-904b20f938e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:17.965071 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:17.970648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39329473-3d5a-43d1-8559-96d6c4a4b9e2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:18.013628 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86911a75-af8d-48eb-80d8-af25827c27a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:18.014563 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86911a75-af8d-48eb-80d8-af25827c27a9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:18.014993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86911a75-af8d-48eb-80d8-af25827c27a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:18.015414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86911a75-af8d-48eb-80d8-af25827c27a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:18.016391 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86911a75-af8d-48eb-80d8-af25827c27a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:18.016876 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86911a75-af8d-48eb-80d8-af25827c27a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:18.018401 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86911a75-af8d-48eb-80d8-af25827c27a9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:18.032569 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 360/736] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:18.033884 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6025f-54d4-49ee-8075-904b20f938e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:18.035499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6025f-54d4-49ee-8075-904b20f938e2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:18.037898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86911a75-af8d-48eb-80d8-af25827c27a9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:18.182471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86911a75-af8d-48eb-80d8-af25827c27a9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:18.220198 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 377/737] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:16:17 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:16:18.226575 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6025f-54d4-49ee-8075-904b20f938e2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:18.228437 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbac91a-8cac-4652-859a-85868ac1ad3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:18.229726 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbac91a-8cac-4652-859a-85868ac1ad3d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:18.230409 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbac91a-8cac-4652-859a-85868ac1ad3d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:18.230809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbac91a-8cac-4652-859a-85868ac1ad3d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:18.240072 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbac91a-8cac-4652-859a-85868ac1ad3d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:18.242341 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbac91a-8cac-4652-859a-85868ac1ad3d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:18.248718 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:18.261698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-fdbac91a-8cac-4652-859a-85868ac1ad3d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:16:18.261698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:18.261698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:18.261698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:18.261698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:18.261698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:18.261698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:18.261698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:18.261698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:18.261698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:18.261698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:18.261698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:18.261698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:18.261698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:18.261698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:18.261698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:18.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:18.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:18.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:18.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:18.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:18.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:18.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:18.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:18.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:18.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:18.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:18.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:18.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:18.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:18.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:18.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:18.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:18.262869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:18.263867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:18.263867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:18.263867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:18.263867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:18.263867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:18.263867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:18.263867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:18.263867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:18.263867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:16:18.263867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:18.266952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 361/738] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:16:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:18.272094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbac91a-8cac-4652-859a-85868ac1ad3d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:18.319896 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:18.399895 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6025f-54d4-49ee-8075-904b20f938e2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:18.425307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbac91a-8cac-4652-859a-85868ac1ad3d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:18.437320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d26db-0e9f-447d-ad9c-f85362628ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:18.438330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d26db-0e9f-447d-ad9c-f85362628ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:18.438810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d26db-0e9f-447d-ad9c-f85362628ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:18.439345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d26db-0e9f-447d-ad9c-f85362628ec8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:18.451145 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d26db-0e9f-447d-ad9c-f85362628ec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:18.453038 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d26db-0e9f-447d-ad9c-f85362628ec8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:18.460751 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c6843-2b79-4523-a1a7-6cefce5c6032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:18.461434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c6843-2b79-4523-a1a7-6cefce5c6032 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:18.461724 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c6843-2b79-4523-a1a7-6cefce5c6032 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:18.462008 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c6843-2b79-4523-a1a7-6cefce5c6032 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:18.463007 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c6843-2b79-4523-a1a7-6cefce5c6032 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:18.463168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c6843-2b79-4523-a1a7-6cefce5c6032 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:18.464653 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c6843-2b79-4523-a1a7-6cefce5c6032 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:18.472526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-281d26db-0e9f-447d-ad9c-f85362628ec8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:18.472526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:18.472526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:18.472526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:18.472526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:18.472526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:18.472526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:18.472526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:18.472526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:18.472526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:18.472526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:18.472526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:18.472526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:18.472526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:18.472526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:18.473651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:18.473651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:18.473651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:18.473651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:18.473651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:18.473651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:18.473651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:18.473651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:18.473651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:18.473651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:18.473651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:18.473651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:18.473651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:18.473651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:18.473651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:18.473651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:18.473651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:18.474613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:18.474613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:18.474613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:18.474613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:18.481682 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 378/739] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:18.483209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c20bd-25fa-4919-8021-c61a3a92d7e3 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:18.483690 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c20bd-25fa-4919-8021-c61a3a92d7e3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:18.484154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c20bd-25fa-4919-8021-c61a3a92d7e3 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:18.485911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 362/740] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Jul 18 10:16:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:18.486509 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 379/741] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:18.491331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c6843-2b79-4523-a1a7-6cefce5c6032 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:18.493968 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327dfb2-8596-4263-b255-4efd7af0e3c3 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:18.494486 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327dfb2-8596-4263-b255-4efd7af0e3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:18.494955 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327dfb2-8596-4263-b255-4efd7af0e3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:18.512159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1327dfb2-8596-4263-b255-4efd7af0e3c3 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:18.513666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1327dfb2-8596-4263-b255-4efd7af0e3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:18.576987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:18.577507 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:18.600544 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 380/742] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 10:16:18.600544 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327dfb2-8596-4263-b255-4efd7af0e3c3 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:18.611059 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c6843-2b79-4523-a1a7-6cefce5c6032 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:18.656147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c752c-c3da-4d8b-98e7-529c85402ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:18.656853 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c752c-c3da-4d8b-98e7-529c85402ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:18.657144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c752c-c3da-4d8b-98e7-529c85402ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:18.657426 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c752c-c3da-4d8b-98e7-529c85402ad4 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:18.658152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c752c-c3da-4d8b-98e7-529c85402ad4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:18.658470 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c752c-c3da-4d8b-98e7-529c85402ad4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:18.660322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c752c-c3da-4d8b-98e7-529c85402ad4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:18.675592 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 363/743] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:18.676620 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c752c-c3da-4d8b-98e7-529c85402ad4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:18.725666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327dfb2-8596-4263-b255-4efd7af0e3c3 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:18.764116 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9d477-c91b-47e5-8959-d5bd5ec88742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:18.765265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9d477-c91b-47e5-8959-d5bd5ec88742 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:18.765773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9d477-c91b-47e5-8959-d5bd5ec88742 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:18.766314 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9d477-c91b-47e5-8959-d5bd5ec88742 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:18.803422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c752c-c3da-4d8b-98e7-529c85402ad4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:18.837673 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:18.848870 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9d477-c91b-47e5-8959-d5bd5ec88742 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:18.850603 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d8678-0a83-4f64-b39f-cc33e2733667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:18.850870 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9d477-c91b-47e5-8959-d5bd5ec88742 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:18.851331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d8678-0a83-4f64-b39f-cc33e2733667 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:18.851620 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d8678-0a83-4f64-b39f-cc33e2733667 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:18.851951 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d8678-0a83-4f64-b39f-cc33e2733667 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:18.852727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d8678-0a83-4f64-b39f-cc33e2733667 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:18.854278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d8678-0a83-4f64-b39f-cc33e2733667 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:18.859373 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:18.880561 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 364/744] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 18 10:16:18 2022] POST /identity/v3/services => generated 240 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:16:18.880561 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d8678-0a83-4f64-b39f-cc33e2733667 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:19.010612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d8678-0a83-4f64-b39f-cc33e2733667 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:19.046277 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 381/745] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:16:18 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 10:16:19.054102 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9d477-c91b-47e5-8959-d5bd5ec88742 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:19.060536 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709d204-bb82-4ce0-9362-fd4481e2a2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:19.060536 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709d204-bb82-4ce0-9362-fd4481e2a2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:19.060536 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709d204-bb82-4ce0-9362-fd4481e2a2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:19.060536 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709d204-bb82-4ce0-9362-fd4481e2a2d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:19.072535 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3709d204-bb82-4ce0-9362-fd4481e2a2d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:19.072535 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3709d204-bb82-4ce0-9362-fd4481e2a2d8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:19.226308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-3709d204-bb82-4ce0-9362-fd4481e2a2d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:19.226308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:19.226308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:19.226308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:19.226308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:19.226308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:19.226308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:19.226308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:19.226308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:19.226308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:19.226308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:19.226308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:19.226308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:19.226308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:19.226308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:19.227304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:19.227304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:19.227304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:19.227304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:19.227304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:19.227304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:19.227304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:19.227304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:19.227304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:19.227304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:19.227304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:19.227304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:19.227304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:19.227304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:19.227304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:19.227304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:19.227304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:19.228208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:19.228208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:19.228208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:19.228208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:19.228208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:19.228208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:19.228208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:19.228208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:19.230375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 365/746] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:19.235413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709d204-bb82-4ce0-9362-fd4481e2a2d8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:19.250269 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:19.261218 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9d477-c91b-47e5-8959-d5bd5ec88742 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:19.298899 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47841c-5100-4a45-8633-fecbdfc783e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:19.299793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47841c-5100-4a45-8633-fecbdfc783e2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:19.300071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47841c-5100-4a45-8633-fecbdfc783e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:19.300440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47841c-5100-4a45-8633-fecbdfc783e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:19.304323 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:19.314848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47841c-5100-4a45-8633-fecbdfc783e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:19.316645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47841c-5100-4a45-8633-fecbdfc783e2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:19.341545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-eb47841c-5100-4a45-8633-fecbdfc783e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:19.341545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:19.341545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:19.341545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:19.341545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:19.341545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:19.341545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:19.341545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:19.341545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:19.341545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:19.341545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:19.341545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:19.341545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:19.341545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:19.341545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:19.342676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:19.342676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:19.342676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:19.342676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:19.342676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:19.342676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:19.342676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:19.342676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:19.342676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:19.342676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:19.342676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:19.342676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:19.342676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:19.342676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:19.342676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:19.342676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:19.342676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:19.343801 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:19.343801 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:19.343801 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:19.343801 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:19.346398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 382/747] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:19.352885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47841c-5100-4a45-8633-fecbdfc783e2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:19.362373 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709d204-bb82-4ce0-9362-fd4481e2a2d8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:19.399525 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6efddb9-099a-4f09-b16f-b696fbda5d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:19.400708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efddb9-099a-4f09-b16f-b696fbda5d11 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:19.401950 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efddb9-099a-4f09-b16f-b696fbda5d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:19.402444 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efddb9-099a-4f09-b16f-b696fbda5d11 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:19.403422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6efddb9-099a-4f09-b16f-b696fbda5d11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:19.403954 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6efddb9-099a-4f09-b16f-b696fbda5d11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:19.405652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6efddb9-099a-4f09-b16f-b696fbda5d11 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:19.420188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 366/748] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:19.430080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6efddb9-099a-4f09-b16f-b696fbda5d11 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:19.536005 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47841c-5100-4a45-8633-fecbdfc783e2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:19.606951 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ae8ab-3537-484f-8613-2862bc132cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:19.608009 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ae8ab-3537-484f-8613-2862bc132cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:19.608525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ae8ab-3537-484f-8613-2862bc132cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:19.609022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ae8ab-3537-484f-8613-2862bc132cc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:19.609964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ae8ab-3537-484f-8613-2862bc132cc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:19.610467 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ae8ab-3537-484f-8613-2862bc132cc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:19.612048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ae8ab-3537-484f-8613-2862bc132cc4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:19.631455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6efddb9-099a-4f09-b16f-b696fbda5d11 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:19.638157 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 383/749] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:19.643947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ae8ab-3537-484f-8613-2862bc132cc4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:19.658873 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:19.668529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:19.706870 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7888b-169b-4c4f-93a1-ae2db9a4ff4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:19.707457 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7888b-169b-4c4f-93a1-ae2db9a4ff4c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:19.707741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7888b-169b-4c4f-93a1-ae2db9a4ff4c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:19.708018 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7888b-169b-4c4f-93a1-ae2db9a4ff4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:19.708888 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7888b-169b-4c4f-93a1-ae2db9a4ff4c None admin] RBAC: Adding query filter params (role.id=f57b3e871ce142a090d6aa7dea9ac7d5, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=a09029144c454b099bea86aaa5229ef0) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:19.709203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7888b-169b-4c4f-93a1-ae2db9a4ff4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:19.711183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7888b-169b-4c4f-93a1-ae2db9a4ff4c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:19.727777 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 367/750] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:16:19 2022] GET /identity/v3/role_assignments?user.id=a09029144c454b099bea86aaa5229ef0&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=f57b3e871ce142a090d6aa7dea9ac7d5 => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:19.819286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ae8ab-3537-484f-8613-2862bc132cc4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:19.858106 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a7cf7-28a3-4301-aa73-d70c66b47153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:19.858918 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a7cf7-28a3-4301-aa73-d70c66b47153 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:19.859202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a7cf7-28a3-4301-aa73-d70c66b47153 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:19.859475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a7cf7-28a3-4301-aa73-d70c66b47153 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:19.867235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a7cf7-28a3-4301-aa73-d70c66b47153 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:19.868808 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a7cf7-28a3-4301-aa73-d70c66b47153 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:19.885049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-592a7cf7-28a3-4301-aa73-d70c66b47153 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:19.885049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:19.885049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:19.885049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:19.885049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:19.885049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:19.885049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:19.885049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:19.885049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:19.885049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:19.885049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:19.885049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:19.885049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:19.885049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:19.885049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:19.886040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:19.886040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:19.886040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:19.886040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:19.886040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:19.886040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:19.886040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:19.886040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:19.886040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:19.886040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:19.886040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:19.886040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:19.886040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:19.886040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:19.886040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:19.886040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:19.886040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:19.886934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:19.886934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:19.886934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:19.886934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:19.886934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:19.886934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:19.886934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:19.886934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:19.886934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 384/751] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:19.892099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7888b-169b-4c4f-93a1-ae2db9a4ff4c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:20.047138 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7888b-169b-4c4f-93a1-ae2db9a4ff4c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:20.088583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f8315-ea8b-4ef9-a9db-7932206646bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:20.089257 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f8315-ea8b-4ef9-a9db-7932206646bf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:20.089540 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f8315-ea8b-4ef9-a9db-7932206646bf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:20.089816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f8315-ea8b-4ef9-a9db-7932206646bf None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:20.090612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f8315-ea8b-4ef9-a9db-7932206646bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:20.090923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f8315-ea8b-4ef9-a9db-7932206646bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:20.092362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f8315-ea8b-4ef9-a9db-7932206646bf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:20.104471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 368/752] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:20.110717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a7cf7-28a3-4301-aa73-d70c66b47153 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:20.180900 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccaa15-fe1d-4ed9-9b2e-9559f900d9b5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:20.181213 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccaa15-fe1d-4ed9-9b2e-9559f900d9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:20.181455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccaa15-fe1d-4ed9-9b2e-9559f900d9b5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:20.183182 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 369/753] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:16:20.188376 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14faf3c5-07bb-4fd0-a78e-921eb43754bf None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:20.188694 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14faf3c5-07bb-4fd0-a78e-921eb43754bf None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:20.188980 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14faf3c5-07bb-4fd0-a78e-921eb43754bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:20.204749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:20.218685 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-14faf3c5-07bb-4fd0-a78e-921eb43754bf None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:20.220480 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-14faf3c5-07bb-4fd0-a78e-921eb43754bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:20.251944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a7cf7-28a3-4301-aa73-d70c66b47153 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:20.253699 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:20.292212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 370/754] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:16:20.293569 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59648d-e184-4143-a38e-9707d1e35f11 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:20.293858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59648d-e184-4143-a38e-9707d1e35f11 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:20.294136 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59648d-e184-4143-a38e-9707d1e35f11 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:20.295631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346e1b9-0db1-4570-a488-4014987f6df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:20.296328 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346e1b9-0db1-4570-a488-4014987f6df8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:20.298292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346e1b9-0db1-4570-a488-4014987f6df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:20.298532 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346e1b9-0db1-4570-a488-4014987f6df8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:20.299441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 371/755] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:20.299608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346e1b9-0db1-4570-a488-4014987f6df8 None admin] RBAC: Adding query filter params (role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=fd6168873f2846c8af73b903bba17eb8) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:20.299848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346e1b9-0db1-4570-a488-4014987f6df8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:20.300829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea12d0-9806-4c7f-83c0-4ffa120073be None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:20.301177 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea12d0-9806-4c7f-83c0-4ffa120073be None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:20.301362 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346e1b9-0db1-4570-a488-4014987f6df8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:20.301569 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea12d0-9806-4c7f-83c0-4ffa120073be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:20.316289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 385/756] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:16:20 2022] GET /identity/v3/role_assignments?user.id=fd6168873f2846c8af73b903bba17eb8&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:20.330613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346e1b9-0db1-4570-a488-4014987f6df8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:20.333508 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b1ea12d0-9806-4c7f-83c0-4ffa120073be None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:20.334727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b1ea12d0-9806-4c7f-83c0-4ffa120073be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:20.462231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 372/757] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:16:20.488016 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea12d0-9806-4c7f-83c0-4ffa120073be None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:20.531443 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346e1b9-0db1-4570-a488-4014987f6df8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:20.586218 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:20.613228 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074325d-45a4-4aec-817c-31e4a456b3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:20.614041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074325d-45a4-4aec-817c-31e4a456b3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:20.614337 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074325d-45a4-4aec-817c-31e4a456b3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:20.614617 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074325d-45a4-4aec-817c-31e4a456b3c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:20.622030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:20.629458 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea12d0-9806-4c7f-83c0-4ffa120073be None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:20.644678 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074325d-45a4-4aec-817c-31e4a456b3c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:20.656645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9074325d-45a4-4aec-817c-31e4a456b3c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:16:20.656645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:20.656645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:20.656645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:20.656645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:20.656645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:20.656645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:20.656645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:20.656645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:20.656645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:20.656645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:20.656645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:20.656645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:20.656645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:20.656645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:20.656645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:20.657623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:20.657623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:20.657623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:20.657623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:20.657623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:20.657623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:20.657623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:20.657623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:20.657623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:20.657623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:20.657623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:20.657623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:20.657623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:20.657623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:20.657623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:20.657623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:20.658489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:20.658489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:20.658489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:20.658489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:20.658489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:20.658489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:20.658489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:20.658489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:20.658489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:20.658489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:20.658489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:16:20.658489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:20.661291 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 386/758] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:16:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:20.669434 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074325d-45a4-4aec-817c-31e4a456b3c1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:20.680308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08154ae1-340c-4321-bf30-81dc66d1a676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:20.685486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08154ae1-340c-4321-bf30-81dc66d1a676 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:20.685926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08154ae1-340c-4321-bf30-81dc66d1a676 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:20.686213 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08154ae1-340c-4321-bf30-81dc66d1a676 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:20.686888 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08154ae1-340c-4321-bf30-81dc66d1a676 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:20.688256 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08154ae1-340c-4321-bf30-81dc66d1a676 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:20.718145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-08154ae1-340c-4321-bf30-81dc66d1a676 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:16:20.718145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:20.718145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:20.718145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:20.718145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:20.718145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:20.718145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:20.718145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:20.718145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:16:20.718145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:20.718145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:20.718145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:20.718145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:20.718145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:20.718145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:20.718145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:20.719122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:20.719122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:20.719122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:20.719122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:20.719122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:20.719122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:20.719122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:20.719122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:20.719122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:20.719122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:20.719122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:20.719122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:20.719122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:20.719122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:20.719122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:20.719122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:20.719968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:20.719968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:20.719968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:20.719968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:20.719968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:20.719968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:20.719968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:20.719968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:20.719968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:20.719968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:20.719968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:20.719968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:20.719968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:20.719968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:16:20.719968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:20.719968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 373/759] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Jul 18 10:16:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:20.729040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08154ae1-340c-4321-bf30-81dc66d1a676 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:20.788013 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074325d-45a4-4aec-817c-31e4a456b3c1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:20.832280 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f65cc-fdfc-46ac-b973-2e1520eabc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:20.832982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f65cc-fdfc-46ac-b973-2e1520eabc18 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:20.833266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f65cc-fdfc-46ac-b973-2e1520eabc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:20.833540 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f65cc-fdfc-46ac-b973-2e1520eabc18 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:20.834297 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f65cc-fdfc-46ac-b973-2e1520eabc18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:20.834603 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f65cc-fdfc-46ac-b973-2e1520eabc18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:20.837222 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f65cc-fdfc-46ac-b973-2e1520eabc18 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:20.846738 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 387/760] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:16:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:20.852236 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f65cc-fdfc-46ac-b973-2e1520eabc18 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:20.923333 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08154ae1-340c-4321-bf30-81dc66d1a676 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:20.970884 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:20.973335 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f573f2-3f97-477b-9635-52564647b67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:20.974027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f573f2-3f97-477b-9635-52564647b67b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:20.974314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f573f2-3f97-477b-9635-52564647b67b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:20.975793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f573f2-3f97-477b-9635-52564647b67b None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:20.975793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f573f2-3f97-477b-9635-52564647b67b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:20.975793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f573f2-3f97-477b-9635-52564647b67b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:20.977719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f573f2-3f97-477b-9635-52564647b67b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:20.990777 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 374/761] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Jul 18 10:16:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:20.996314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f573f2-3f97-477b-9635-52564647b67b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:21.032134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f65cc-fdfc-46ac-b973-2e1520eabc18 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:21.070577 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:21.077143 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315bb3c-81e9-4c59-a333-9658ba678407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:21.077953 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315bb3c-81e9-4c59-a333-9658ba678407 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:21.078239 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315bb3c-81e9-4c59-a333-9658ba678407 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:21.078512 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315bb3c-81e9-4c59-a333-9658ba678407 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:21.106170 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315bb3c-81e9-4c59-a333-9658ba678407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:21.107568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315bb3c-81e9-4c59-a333-9658ba678407 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:21.144344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-2315bb3c-81e9-4c59-a333-9658ba678407 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:21.144344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:21.144344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:21.144344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:21.144344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:21.144344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:21.144344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:21.144344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:21.144344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:21.144344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:21.144344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:21.144344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:21.144344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:21.144344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:21.144344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:21.145339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:21.145339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:21.145339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:21.145339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:21.145339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:21.145339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:21.145339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:21.145339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:21.145339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:21.145339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:21.145339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:21.145339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:21.145339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:21.145339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:21.145339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:21.145339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:21.145339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:21.146243 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:21.146243 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:21.146243 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:21.146243 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:21.156254 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 388/762] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:21.164628 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315bb3c-81e9-4c59-a333-9658ba678407 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:21.254704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f573f2-3f97-477b-9635-52564647b67b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:21.278330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315bb3c-81e9-4c59-a333-9658ba678407 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:21.279909 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:21.321706 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd42b13-513a-412f-9889-8b95e4863af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:21.322373 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd42b13-513a-412f-9889-8b95e4863af6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:21.322660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd42b13-513a-412f-9889-8b95e4863af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:21.322940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd42b13-513a-412f-9889-8b95e4863af6 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:21.324913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd42b13-513a-412f-9889-8b95e4863af6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:21.325226 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd42b13-513a-412f-9889-8b95e4863af6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:21.326945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd42b13-513a-412f-9889-8b95e4863af6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:21.340207 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 389/763] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:21.345948 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd42b13-513a-412f-9889-8b95e4863af6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:21.369231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc8bf2-0a3f-49bf-ab4f-fd9ffcb3d2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:21.369917 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc8bf2-0a3f-49bf-ab4f-fd9ffcb3d2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:21.370204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc8bf2-0a3f-49bf-ab4f-fd9ffcb3d2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:21.370482 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc8bf2-0a3f-49bf-ab4f-fd9ffcb3d2b5 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:21.371195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc8bf2-0a3f-49bf-ab4f-fd9ffcb3d2b5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:21.371502 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc8bf2-0a3f-49bf-ab4f-fd9ffcb3d2b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:21.372884 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc8bf2-0a3f-49bf-ab4f-fd9ffcb3d2b5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:21.386811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 375/764] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Jul 18 10:16:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:21.392469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc8bf2-0a3f-49bf-ab4f-fd9ffcb3d2b5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:21.458096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd42b13-513a-412f-9889-8b95e4863af6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:21.509146 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929fd4d-7bb0-4cc2-a4ca-8e34b15888cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:21.509946 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929fd4d-7bb0-4cc2-a4ca-8e34b15888cb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:21.510228 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929fd4d-7bb0-4cc2-a4ca-8e34b15888cb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:21.510501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929fd4d-7bb0-4cc2-a4ca-8e34b15888cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:21.523882 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929fd4d-7bb0-4cc2-a4ca-8e34b15888cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:21.527012 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929fd4d-7bb0-4cc2-a4ca-8e34b15888cb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:21.545770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7929fd4d-7bb0-4cc2-a4ca-8e34b15888cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:16:21.545770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:21.545770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:21.545770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:21.545770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:21.545770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:21.545770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:21.545770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:21.545770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:21.545770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:21.545770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:21.545770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:21.545770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:21.545770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:21.545770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:21.545770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:21.546788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:21.546788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:21.546788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:21.546788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:21.546788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:21.546788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:21.546788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:21.546788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:21.546788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:21.546788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:21.546788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:21.546788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:21.546788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:21.546788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:21.546788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:21.546788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:21.546788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:21.546788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:21.547733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:21.547733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:21.547733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:21.547733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:21.547733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:21.547733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:21.547733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:21.547733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:21.547733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:16:21.547733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:21.548278 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 390/765] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jul 18 10:16:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:21.554163 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929fd4d-7bb0-4cc2-a4ca-8e34b15888cb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:21.577746 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:21.590328 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:21.591829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc8bf2-0a3f-49bf-ab4f-fd9ffcb3d2b5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:21.635184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df1dc3-1671-4189-82f7-b4c6b1df3021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:21.636166 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df1dc3-1671-4189-82f7-b4c6b1df3021 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:21.636694 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df1dc3-1671-4189-82f7-b4c6b1df3021 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:21.637156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df1dc3-1671-4189-82f7-b4c6b1df3021 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:21.638135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df1dc3-1671-4189-82f7-b4c6b1df3021 None admin] RBAC: Adding query filter params (interface=public, service_id=4916961930bf474d9037cb179d65c57c, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:21.638637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df1dc3-1671-4189-82f7-b4c6b1df3021 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:21.640629 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df1dc3-1671-4189-82f7-b4c6b1df3021 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:21.658420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 376/766] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jul 18 10:16:21 2022] GET /identity/v3/endpoints?service_id=4916961930bf474d9037cb179d65c57c&interface=public®ion_id=RegionOne => generated 194 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:21.660210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc3b63-f521-4451-84e4-78e2e7d96a72 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:21.660727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc3b63-f521-4451-84e4-78e2e7d96a72 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:21.661190 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc3b63-f521-4451-84e4-78e2e7d96a72 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:21.663093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 377/767] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:21.664049 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc3b63-f521-4451-84e4-78e2e7d96a72 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:21.692854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929fd4d-7bb0-4cc2-a4ca-8e34b15888cb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:21.738705 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe851f-76bc-4a08-879c-404fe7c61706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:21.739706 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe851f-76bc-4a08-879c-404fe7c61706 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:21.740040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe851f-76bc-4a08-879c-404fe7c61706 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:21.740328 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe851f-76bc-4a08-879c-404fe7c61706 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:21.741265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe851f-76bc-4a08-879c-404fe7c61706 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:21.741731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe851f-76bc-4a08-879c-404fe7c61706 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:21.744051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe851f-76bc-4a08-879c-404fe7c61706 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:21.773056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 391/768] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jul 18 10:16:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:21.774905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09b850-815f-45c0-b6a0-4fc2beb512cd None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:21.775233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09b850-815f-45c0-b6a0-4fc2beb512cd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:21.775490 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09b850-815f-45c0-b6a0-4fc2beb512cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:21.787399 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc3b63-f521-4451-84e4-78e2e7d96a72 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:21.793080 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ca09b850-815f-45c0-b6a0-4fc2beb512cd None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:21.794280 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ca09b850-815f-45c0-b6a0-4fc2beb512cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:21.832703 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e5286-6913-4d5e-9e1f-3e0955141f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:21.834977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e5286-6913-4d5e-9e1f-3e0955141f2e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:21.835494 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e5286-6913-4d5e-9e1f-3e0955141f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:21.835971 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e5286-6913-4d5e-9e1f-3e0955141f2e None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:21.836962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e5286-6913-4d5e-9e1f-3e0955141f2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:21.838608 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e5286-6913-4d5e-9e1f-3e0955141f2e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:21.846584 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 378/769] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jul 18 10:16:21 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:21.848157 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e5286-6913-4d5e-9e1f-3e0955141f2e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:21.902383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:21.904436 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 392/770] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:21.947640 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca09b850-815f-45c0-b6a0-4fc2beb512cd None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:21.964030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:22.035961 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e5286-6913-4d5e-9e1f-3e0955141f2e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:22.084518 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d635d-b712-4a02-942d-ff8b3e452e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:22.086181 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d635d-b712-4a02-942d-ff8b3e452e55 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:22.086472 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d635d-b712-4a02-942d-ff8b3e452e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:22.086760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d635d-b712-4a02-942d-ff8b3e452e55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:22.088367 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca09b850-815f-45c0-b6a0-4fc2beb512cd None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:22.098911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d635d-b712-4a02-942d-ff8b3e452e55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:22.100401 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d635d-b712-4a02-942d-ff8b3e452e55 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:22.124789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-585d635d-b712-4a02-942d-ff8b3e452e55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:22.124789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:22.124789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:22.124789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:22.124789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:22.124789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:22.124789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:22.124789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:22.124789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:22.124789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:22.124789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:22.124789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:22.124789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:22.124789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:22.124789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:22.125761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:22.125761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:22.125761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:22.125761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:22.125761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:22.125761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:22.125761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:22.125761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:22.125761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:22.125761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:22.125761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:22.125761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:22.125761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:22.125761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:22.125761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:22.125761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:22.125761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:22.127280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:22.127280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:22.127280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:22.127280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:22.127280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 379/771] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:22.127728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4711a8-e400-43c6-b57a-0f432a357daf None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:22.128022 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4711a8-e400-43c6-b57a-0f432a357daf None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:22.128305 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4711a8-e400-43c6-b57a-0f432a357daf None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:22.134180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 380/772] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:22.134180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4711a8-e400-43c6-b57a-0f432a357daf None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:22.144938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1fa9b-81e0-4c89-9adf-d49a9cd3b56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:22.146552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1fa9b-81e0-4c89-9adf-d49a9cd3b56b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:22.147925 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1fa9b-81e0-4c89-9adf-d49a9cd3b56b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:22.148411 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1fa9b-81e0-4c89-9adf-d49a9cd3b56b None admin] PATH_INFO: `/v3/services/network` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:22.149556 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb1fa9b-81e0-4c89-9adf-d49a9cd3b56b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:22.151152 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb1fa9b-81e0-4c89-9adf-d49a9cd3b56b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:22.170806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-2bb1fa9b-81e0-4c89-9adf-d49a9cd3b56b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:16:22.170806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:22.170806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:22.170806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:22.170806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:22.170806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:22.170806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:22.170806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:22.170806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:16:22.170806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:22.170806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:22.170806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:22.170806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:22.170806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:22.170806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:22.170806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:22.170806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:22.171864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:22.171864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:22.171864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:22.171864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:22.171864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:22.171864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:22.171864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:22.171864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:22.171864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:22.171864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:22.171864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:22.171864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:22.171864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:22.171864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:22.171864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:22.171864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:22.172750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:22.172750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:22.172750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:22.172750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:22.172750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:22.172750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:22.172750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:22.172750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:22.172750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:22.172750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:22.172750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:22.172750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:22.172750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:16:22.172750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:22.172750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 393/773] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:21 2022] GET /identity/v3/services/network => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:22.174084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff044c7e-064d-43f2-a46d-aba48c177d78 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:22.174084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff044c7e-064d-43f2-a46d-aba48c177d78 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:22.174338 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff044c7e-064d-43f2-a46d-aba48c177d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:22.199974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ff044c7e-064d-43f2-a46d-aba48c177d78 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:22.202047 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ff044c7e-064d-43f2-a46d-aba48c177d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:22.232467 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:22.278381 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4711a8-e400-43c6-b57a-0f432a357daf None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:22.305147 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 394/774] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:22.308689 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff044c7e-064d-43f2-a46d-aba48c177d78 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:22.312490 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:22.344406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c819bd-63da-48e5-8f44-b13e6e032f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:22.345393 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c819bd-63da-48e5-8f44-b13e6e032f2b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:22.345507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c819bd-63da-48e5-8f44-b13e6e032f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:22.345783 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c819bd-63da-48e5-8f44-b13e6e032f2b None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:22.346593 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c819bd-63da-48e5-8f44-b13e6e032f2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:22.346831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c819bd-63da-48e5-8f44-b13e6e032f2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:22.348210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c819bd-63da-48e5-8f44-b13e6e032f2b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:22.362794 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 381/775] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:22.363926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c819bd-63da-48e5-8f44-b13e6e032f2b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:22.435273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff044c7e-064d-43f2-a46d-aba48c177d78 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:22.473295 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ea517-e97d-47b0-8fb0-7886035418fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:22.474115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ea517-e97d-47b0-8fb0-7886035418fc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:22.474390 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ea517-e97d-47b0-8fb0-7886035418fc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:22.474790 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ea517-e97d-47b0-8fb0-7886035418fc None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:22.475668 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ea517-e97d-47b0-8fb0-7886035418fc None admin] RBAC: Adding query filter params (name=network) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:22.476170 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ea517-e97d-47b0-8fb0-7886035418fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:22.477521 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ea517-e97d-47b0-8fb0-7886035418fc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:22.483144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 395/776] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 18 10:16:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:22.484584 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ea517-e97d-47b0-8fb0-7886035418fc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:22.501661 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c819bd-63da-48e5-8f44-b13e6e032f2b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:22.538199 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:22.547724 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261cd6bc-8fbf-4ec2-956a-3f80198d5ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:22.548836 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cd6bc-8fbf-4ec2-956a-3f80198d5ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:22.549308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cd6bc-8fbf-4ec2-956a-3f80198d5ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:22.549793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cd6bc-8fbf-4ec2-956a-3f80198d5ca8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:22.558413 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:22.562117 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261cd6bc-8fbf-4ec2-956a-3f80198d5ca8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:22.571029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-261cd6bc-8fbf-4ec2-956a-3f80198d5ca8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:22.571029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:22.571029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:22.571029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:22.571029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:22.571029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:22.571029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:22.571029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:22.571029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:22.571029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:22.571029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:22.571029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:22.571029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:22.571029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:22.571029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:22.571029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:22.572005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:22.572005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:22.572005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:22.572005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:22.572005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:22.572005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:22.572005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:22.572005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:22.572005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:22.572005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:22.572005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:22.572005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:22.572005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:22.572005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:22.572005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:22.572005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:22.572915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:22.572915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:22.572915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:22.572915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:22.572915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:22.572915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:22.572915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:22.572915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:22.572915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:22.572915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:22.572915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:22.572915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:22.575606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 382/777] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:16:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:22.576653 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261cd6bc-8fbf-4ec2-956a-3f80198d5ca8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:22.607265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ea517-e97d-47b0-8fb0-7886035418fc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:22.644270 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaef078-ff5c-4919-894a-d627f916a979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:22.645468 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaef078-ff5c-4919-894a-d627f916a979 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:22.645911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaef078-ff5c-4919-894a-d627f916a979 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:22.646341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaef078-ff5c-4919-894a-d627f916a979 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:22.656373 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaef078-ff5c-4919-894a-d627f916a979 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:22.657886 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaef078-ff5c-4919-894a-d627f916a979 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:22.673355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-cdaef078-ff5c-4919-894a-d627f916a979 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:22.673355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:22.673355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:22.673355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:22.673355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:22.673355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:22.673355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:22.673355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:22.673355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:22.673355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:22.673355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:22.673355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:22.673355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:22.673355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:22.673355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:22.674514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:22.674514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:22.674514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:22.674514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:22.674514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:22.674514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:22.674514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:22.674514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:22.674514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:22.674514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:22.674514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:22.674514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:22.674514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:22.674514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:22.674514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:22.674514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:22.674514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:22.675458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:22.675458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:22.675458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:22.675458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:22.675458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:22.675458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:22.675458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:22.675458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:22.675458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 396/778] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:22.675458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaef078-ff5c-4919-894a-d627f916a979 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:22.706527 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261cd6bc-8fbf-4ec2-956a-3f80198d5ca8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:22.742791 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b87154-2890-499b-bff8-4fb86eb01cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:22.743452 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b87154-2890-499b-bff8-4fb86eb01cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:22.743743 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b87154-2890-499b-bff8-4fb86eb01cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:22.744027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b87154-2890-499b-bff8-4fb86eb01cc3 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:22.744766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b87154-2890-499b-bff8-4fb86eb01cc3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:22.745080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b87154-2890-499b-bff8-4fb86eb01cc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:22.746454 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b87154-2890-499b-bff8-4fb86eb01cc3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:22.752000 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 383/779] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 18 10:16:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:22.753283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20047bc5-f685-43ba-9d89-489d8f4cf3c5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:22.753632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20047bc5-f685-43ba-9d89-489d8f4cf3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:22.753915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20047bc5-f685-43ba-9d89-489d8f4cf3c5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:22.755146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 384/780] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:22.755899 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20047bc5-f685-43ba-9d89-489d8f4cf3c5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:22.785923 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaef078-ff5c-4919-894a-d627f916a979 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:22.814109 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:22.816972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:22.822662 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a58fc-5e12-40ca-ad6f-11acd9973301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:22.823328 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a58fc-5e12-40ca-ad6f-11acd9973301 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:22.823664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a58fc-5e12-40ca-ad6f-11acd9973301 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:22.823967 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a58fc-5e12-40ca-ad6f-11acd9973301 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:22.824751 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a58fc-5e12-40ca-ad6f-11acd9973301 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:22.825066 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a58fc-5e12-40ca-ad6f-11acd9973301 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:22.826452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a58fc-5e12-40ca-ad6f-11acd9973301 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:22.835301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 397/781] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:16:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:22.836457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a58fc-5e12-40ca-ad6f-11acd9973301 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:22.875612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20047bc5-f685-43ba-9d89-489d8f4cf3c5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:22.911012 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e3fe8-61a2-470e-af8c-84f3b43a5ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:22.911768 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e3fe8-61a2-470e-af8c-84f3b43a5ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:22.912053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e3fe8-61a2-470e-af8c-84f3b43a5ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:22.912330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e3fe8-61a2-470e-af8c-84f3b43a5ce5 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:22.913661 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e3fe8-61a2-470e-af8c-84f3b43a5ce5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:22.913972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e3fe8-61a2-470e-af8c-84f3b43a5ce5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:22.915393 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e3fe8-61a2-470e-af8c-84f3b43a5ce5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:22.927606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 385/782] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:22.933530 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558c903-f0ea-4d01-84d4-39f677a9f8d7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:22.933811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558c903-f0ea-4d01-84d4-39f677a9f8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:22.934092 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558c903-f0ea-4d01-84d4-39f677a9f8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:22.946405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a58fc-5e12-40ca-ad6f-11acd9973301 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:22.950668 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b558c903-f0ea-4d01-84d4-39f677a9f8d7 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:22.951852 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b558c903-f0ea-4d01-84d4-39f677a9f8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:22.991996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80d157-f151-478d-bbf1-ee3f9af25fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:22.992618 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80d157-f151-478d-bbf1-ee3f9af25fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:22.992908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80d157-f151-478d-bbf1-ee3f9af25fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:22.993180 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80d157-f151-478d-bbf1-ee3f9af25fb1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:22.993986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb80d157-f151-478d-bbf1-ee3f9af25fb1 None admin] RBAC: Adding query filter params (interface=public, service_id=79901ddedfd04106a8ad513d3080b364, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:22.994306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb80d157-f151-478d-bbf1-ee3f9af25fb1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:22.995686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb80d157-f151-478d-bbf1-ee3f9af25fb1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:23.008475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 398/783] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jul 18 10:16:22 2022] GET /identity/v3/endpoints?service_id=79901ddedfd04106a8ad513d3080b364&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:23.008475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80d157-f151-478d-bbf1-ee3f9af25fb1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:23.032834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 386/784] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:23.032834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558c903-f0ea-4d01-84d4-39f677a9f8d7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:23.074753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:23.083718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:23.126046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80d157-f151-478d-bbf1-ee3f9af25fb1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:23.140919 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558c903-f0ea-4d01-84d4-39f677a9f8d7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:23.163311 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc25ed-2ca7-46e5-b55e-2ad61d6e238b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:23.164117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc25ed-2ca7-46e5-b55e-2ad61d6e238b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:23.164407 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc25ed-2ca7-46e5-b55e-2ad61d6e238b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:23.164715 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc25ed-2ca7-46e5-b55e-2ad61d6e238b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:23.176548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc25ed-2ca7-46e5-b55e-2ad61d6e238b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:23.178339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc25ed-2ca7-46e5-b55e-2ad61d6e238b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:23.182733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023c737-8d85-4c22-a183-3bc6047440b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:23.183637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023c737-8d85-4c22-a183-3bc6047440b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:23.183919 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023c737-8d85-4c22-a183-3bc6047440b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:23.184193 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023c737-8d85-4c22-a183-3bc6047440b4 None admin] PATH_INFO: `/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/29048c2021e340b081ae44a390583e7a/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:23.186322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023c737-8d85-4c22-a183-3bc6047440b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e71a3acd0adf46a2a35b965a27c4998f, user_id=29048c2021e340b081ae44a390583e7a, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:23.187746 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023c737-8d85-4c22-a183-3bc6047440b4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:23.198018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-31cc25ed-2ca7-46e5-b55e-2ad61d6e238b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:23.198018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:23.198018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:23.198018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:23.198018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:23.198018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:23.198018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:23.198018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:23.198018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:23.198018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:23.198018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:23.198018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:23.198018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:23.198018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:23.198018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:23.199176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:23.199176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:23.199176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:23.199176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:23.199176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:23.199176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:23.199176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:23.199176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:23.199176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:23.199176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:23.199176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:23.199176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:23.199176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:23.199176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:23.199176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:23.199176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:23.199176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:23.200134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:23.200134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:23.200134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:23.200134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:23.200134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 399/785] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:23.200434 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc25ed-2ca7-46e5-b55e-2ad61d6e238b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:23.212722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 387/786] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:16:23 2022] PUT /identity/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/29048c2021e340b081ae44a390583e7a/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:16:23.212722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023c737-8d85-4c22-a183-3bc6047440b4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:23.348263 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc25ed-2ca7-46e5-b55e-2ad61d6e238b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:23.351595 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023c737-8d85-4c22-a183-3bc6047440b4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:23.353598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:23.359527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:23.398968 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a0f42-6c16-4931-b347-2606d6eb10bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:23.399632 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a0f42-6c16-4931-b347-2606d6eb10bc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:23.399917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a0f42-6c16-4931-b347-2606d6eb10bc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:23.400187 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a0f42-6c16-4931-b347-2606d6eb10bc None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:23.400311 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14c5d0-09a5-4690-82d1-d3e84b2bd108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:23.400950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a0f42-6c16-4931-b347-2606d6eb10bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:23.401118 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14c5d0-09a5-4690-82d1-d3e84b2bd108 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:23.401403 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14c5d0-09a5-4690-82d1-d3e84b2bd108 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:23.401681 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14c5d0-09a5-4690-82d1-d3e84b2bd108 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:23.402318 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a0f42-6c16-4931-b347-2606d6eb10bc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:23.410761 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 400/787] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jul 18 10:16:23 2022] GET /identity/v3/services => generated 772 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:23.411939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a0f42-6c16-4931-b347-2606d6eb10bc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:23.414366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14c5d0-09a5-4690-82d1-d3e84b2bd108 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:23.422392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-bc14c5d0-09a5-4690-82d1-d3e84b2bd108 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:16:23.422392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:23.422392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:23.422392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:23.422392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:23.422392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:23.422392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:23.422392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:23.422392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:23.422392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:23.422392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:23.422392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:23.422392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:23.422392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:23.422392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:23.422392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:23.423417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:23.423417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:23.423417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:23.423417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:23.423417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:23.423417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:23.423417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:23.423417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:23.423417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:23.423417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:23.423417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:23.423417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:23.423417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:23.423417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:23.423417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:23.423417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:23.424322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:23.424322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:23.424322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:23.424322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:23.424322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:23.424322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:23.424322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:23.424322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:23.424322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:23.424322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:23.424322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:16:23.424322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:23.424322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 388/788] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:16:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:23.430177 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14c5d0-09a5-4690-82d1-d3e84b2bd108 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:23.546016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a0f42-6c16-4931-b347-2606d6eb10bc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:23.601961 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210863cb-091c-410a-ad47-78810eebee05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:23.602572 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210863cb-091c-410a-ad47-78810eebee05 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:23.602856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210863cb-091c-410a-ad47-78810eebee05 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:23.603187 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210863cb-091c-410a-ad47-78810eebee05 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:23.603935 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210863cb-091c-410a-ad47-78810eebee05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:23.605030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210863cb-091c-410a-ad47-78810eebee05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:23.606213 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210863cb-091c-410a-ad47-78810eebee05 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:23.622034 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14c5d0-09a5-4690-82d1-d3e84b2bd108 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:23.626483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 401/789] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:23.633146 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210863cb-091c-410a-ad47-78810eebee05 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:23.664464 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ece8f-1c7b-4ca8-903c-ae66ae6f32ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:23.665149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ece8f-1c7b-4ca8-903c-ae66ae6f32ac None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:23.665431 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ece8f-1c7b-4ca8-903c-ae66ae6f32ac None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:23.665705 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ece8f-1c7b-4ca8-903c-ae66ae6f32ac None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:23.666459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ece8f-1c7b-4ca8-903c-ae66ae6f32ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:23.666767 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ece8f-1c7b-4ca8-903c-ae66ae6f32ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:23.668146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ece8f-1c7b-4ca8-903c-ae66ae6f32ac None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:23.677908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 389/790] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:16:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:23.684771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:23.685262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ece8f-1c7b-4ca8-903c-ae66ae6f32ac None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:23.713266 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:23.746259 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210863cb-091c-410a-ad47-78810eebee05 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:23.782682 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb80b65-9903-423c-9034-0fbac35b1988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:23.784752 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb80b65-9903-423c-9034-0fbac35b1988 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:23.785381 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb80b65-9903-423c-9034-0fbac35b1988 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:23.787093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb80b65-9903-423c-9034-0fbac35b1988 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:23.802615 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb80b65-9903-423c-9034-0fbac35b1988 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:23.806164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb80b65-9903-423c-9034-0fbac35b1988 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:23.828702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ccb80b65-9903-423c-9034-0fbac35b1988 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:16:23.828702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:23.828702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:23.828702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:23.828702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:23.828702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:23.828702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:23.828702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:23.828702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:23.828702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:23.828702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:23.828702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:23.828702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:23.828702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:23.828702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:23.828702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:23.829947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:23.829947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:23.829947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:23.829947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:23.829947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:23.829947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:23.829947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:23.829947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:23.829947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:23.829947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:23.829947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:23.829947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:23.829947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:23.829947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:23.829947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:23.829947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:23.829947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:23.829947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:23.831126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:23.831126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:23.831126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:23.831126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:23.831126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:23.831126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:23.831126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:23.831126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:23.831126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:16:23.831126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:23.831126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 402/791] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 18 10:16:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:23.841489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb80b65-9903-423c-9034-0fbac35b1988 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:23.846082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ece8f-1c7b-4ca8-903c-ae66ae6f32ac None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:23.886195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e42ba-7307-4daa-8fe0-dd7fb3ff626e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:23.887513 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e42ba-7307-4daa-8fe0-dd7fb3ff626e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:23.888204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e42ba-7307-4daa-8fe0-dd7fb3ff626e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:23.888694 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e42ba-7307-4daa-8fe0-dd7fb3ff626e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:23.900538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e42ba-7307-4daa-8fe0-dd7fb3ff626e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:23.902542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e42ba-7307-4daa-8fe0-dd7fb3ff626e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:23.931471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7c3e42ba-7307-4daa-8fe0-dd7fb3ff626e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:23.931471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:23.931471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:23.931471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:23.931471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:23.931471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:23.931471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:23.931471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:23.931471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:23.931471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:23.931471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:23.931471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:23.931471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:23.931471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:23.931471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:23.932665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:23.932665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:23.932665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:23.932665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:23.932665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:23.932665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:23.932665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:23.932665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:23.932665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:23.932665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:23.932665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:23.932665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:23.932665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:23.932665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:23.932665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:23.932665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:23.932665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:23.933621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:23.933621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:23.933621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:23.933621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:23.933621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 390/792] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:23.938422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e42ba-7307-4daa-8fe0-dd7fb3ff626e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:24.026371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:24.036241 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb80b65-9903-423c-9034-0fbac35b1988 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:24.050792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:24.077342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e42ba-7307-4daa-8fe0-dd7fb3ff626e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:24.079890 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238d1c1-b84a-4db7-89c7-5566cd685333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:24.081225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238d1c1-b84a-4db7-89c7-5566cd685333 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:24.081537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238d1c1-b84a-4db7-89c7-5566cd685333 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:24.081814 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238d1c1-b84a-4db7-89c7-5566cd685333 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:24.082672 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238d1c1-b84a-4db7-89c7-5566cd685333 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:24.082980 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238d1c1-b84a-4db7-89c7-5566cd685333 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:24.084477 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238d1c1-b84a-4db7-89c7-5566cd685333 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:24.106807 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 403/793] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Jul 18 10:16:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:24.108124 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc03576-c42d-4624-a5ab-b8cecd248536 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:24.108412 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc03576-c42d-4624-a5ab-b8cecd248536 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:24.108744 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc03576-c42d-4624-a5ab-b8cecd248536 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:24.110212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 404/794] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:24.111830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc03576-c42d-4624-a5ab-b8cecd248536 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:24.125490 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522cd75-e44f-4cae-ae16-77cf4ae7429b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:24.126579 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522cd75-e44f-4cae-ae16-77cf4ae7429b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:24.126867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522cd75-e44f-4cae-ae16-77cf4ae7429b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:24.127428 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522cd75-e44f-4cae-ae16-77cf4ae7429b None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:24.128166 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522cd75-e44f-4cae-ae16-77cf4ae7429b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:24.128486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522cd75-e44f-4cae-ae16-77cf4ae7429b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:24.130629 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522cd75-e44f-4cae-ae16-77cf4ae7429b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:24.148060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 391/795] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:24.149627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100b606-6b8c-426f-9e18-41146826784d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:24.154820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100b606-6b8c-426f-9e18-41146826784d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:24.154820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100b606-6b8c-426f-9e18-41146826784d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:24.171941 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d100b606-6b8c-426f-9e18-41146826784d None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:24.173701 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d100b606-6b8c-426f-9e18-41146826784d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:24.228656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc03576-c42d-4624-a5ab-b8cecd248536 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:24.267954 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 392/796] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:24.269052 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100b606-6b8c-426f-9e18-41146826784d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:24.364849 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:24.388472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4a349-1127-4d9d-b5e0-c88db133e8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:24.389287 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4a349-1127-4d9d-b5e0-c88db133e8e9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:24.389671 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4a349-1127-4d9d-b5e0-c88db133e8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:24.389946 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4a349-1127-4d9d-b5e0-c88db133e8e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:24.401885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4a349-1127-4d9d-b5e0-c88db133e8e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:24.403354 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4a349-1127-4d9d-b5e0-c88db133e8e9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:24.409119 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100b606-6b8c-426f-9e18-41146826784d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:24.413539 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:24.423569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8ca4a349-1127-4d9d-b5e0-c88db133e8e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:24.423569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:24.423569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:24.423569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:24.423569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:24.423569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:24.423569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:24.423569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:24.423569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:24.423569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:24.423569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:24.423569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:24.423569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:24.423569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:24.423569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:24.424692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:24.424692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:24.424692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:24.424692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:24.424692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:24.424692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:24.424692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:24.424692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:24.424692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:24.424692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:24.424692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:24.424692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:24.424692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:24.424692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:24.424692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:24.424692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:24.424692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:24.425603 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:24.425603 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:24.425603 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:24.425603 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:24.428795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 405/797] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:24.429457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4a349-1127-4d9d-b5e0-c88db133e8e9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:24.465842 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cffc0-f317-485e-9146-fadc0839ad21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:24.466983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cffc0-f317-485e-9146-fadc0839ad21 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:24.467671 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cffc0-f317-485e-9146-fadc0839ad21 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:24.468772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cffc0-f317-485e-9146-fadc0839ad21 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:24.476082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1cffc0-f317-485e-9146-fadc0839ad21 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:24.477665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1cffc0-f317-485e-9146-fadc0839ad21 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:24.492887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4a1cffc0-f317-485e-9146-fadc0839ad21 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:16:24.492887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:24.492887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:24.492887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:24.492887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:24.492887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:24.492887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:24.492887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:24.492887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:24.492887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:24.492887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:24.492887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:24.492887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:24.492887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:24.492887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:24.492887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:24.493840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:24.493840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:24.493840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:24.493840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:24.493840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:24.493840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:24.493840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:24.493840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:24.493840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:24.493840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:24.493840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:24.493840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:24.493840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:24.493840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:24.493840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:24.493840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:24.493840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:24.493840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:24.494815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:24.494815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:24.494815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:24.494815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:24.494815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:24.494815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:24.494815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:24.494815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:24.494815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:16:24.494815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:24.498820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 393/798] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:16:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:24.499584 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cffc0-f317-485e-9146-fadc0839ad21 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:24.551103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4a349-1127-4d9d-b5e0-c88db133e8e9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:24.590427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5be480-506a-4fe1-8323-8f5de893b673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:24.591253 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5be480-506a-4fe1-8323-8f5de893b673 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:24.591536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5be480-506a-4fe1-8323-8f5de893b673 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:24.591808 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5be480-506a-4fe1-8323-8f5de893b673 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:24.592792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5be480-506a-4fe1-8323-8f5de893b673 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:24.594047 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5be480-506a-4fe1-8323-8f5de893b673 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:24.609078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9f5be480-506a-4fe1-8323-8f5de893b673 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:16:24.609078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:24.609078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:24.609078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:24.609078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:24.609078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:24.609078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:24.609078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:24.609078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:16:24.609078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:24.609078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:24.609078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:24.609078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:24.609078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:24.609078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:24.609078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:24.610064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:24.610064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:24.610064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:24.610064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:24.610064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:24.610064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:24.610064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:24.610064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:24.610064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:24.610064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:24.610064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:24.610064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:24.610064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:24.610064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:24.610064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:24.610064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:24.611006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:24.611006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:24.611006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:24.611006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:24.611006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:24.611006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:24.611006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:24.611006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:24.611006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:24.611006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:24.611006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:24.611006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:24.611006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:24.611006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 10:16:24.611006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:24.615266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 406/799] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Jul 18 10:16:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:24.616821 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5be480-506a-4fe1-8323-8f5de893b673 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:24.624227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cffc0-f317-485e-9146-fadc0839ad21 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:24.658590 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:24.670139 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609fe51-c409-49b9-99d4-182e3b6e4bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:24.671363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609fe51-c409-49b9-99d4-182e3b6e4bec None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:24.672536 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609fe51-c409-49b9-99d4-182e3b6e4bec None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:24.673662 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609fe51-c409-49b9-99d4-182e3b6e4bec None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:24.676191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609fe51-c409-49b9-99d4-182e3b6e4bec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:24.677091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609fe51-c409-49b9-99d4-182e3b6e4bec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:24.681661 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609fe51-c409-49b9-99d4-182e3b6e4bec None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:24.695902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:24.703563 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 394/800] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:24.704592 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609fe51-c409-49b9-99d4-182e3b6e4bec None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:24.747791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5be480-506a-4fe1-8323-8f5de893b673 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:24.795007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73128b6c-5384-46cc-b6ea-5818a4014feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:24.796354 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73128b6c-5384-46cc-b6ea-5818a4014feb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:24.796878 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73128b6c-5384-46cc-b6ea-5818a4014feb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:24.797513 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73128b6c-5384-46cc-b6ea-5818a4014feb None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:24.798850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73128b6c-5384-46cc-b6ea-5818a4014feb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:24.799366 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73128b6c-5384-46cc-b6ea-5818a4014feb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:24.801471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73128b6c-5384-46cc-b6ea-5818a4014feb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:24.815274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609fe51-c409-49b9-99d4-182e3b6e4bec None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:24.826177 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 407/801] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Jul 18 10:16:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:24.827276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73128b6c-5384-46cc-b6ea-5818a4014feb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:24.854687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f47be3-b99c-4cfb-9946-6ee434419399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:24.855655 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f47be3-b99c-4cfb-9946-6ee434419399 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:24.855936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f47be3-b99c-4cfb-9946-6ee434419399 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:24.856209 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f47be3-b99c-4cfb-9946-6ee434419399 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:24.857005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f47be3-b99c-4cfb-9946-6ee434419399 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:24.857294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f47be3-b99c-4cfb-9946-6ee434419399 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:24.858704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f47be3-b99c-4cfb-9946-6ee434419399 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:24.867993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 395/802] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Jul 18 10:16:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:24.868841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f47be3-b99c-4cfb-9946-6ee434419399 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:24.932420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:24.949652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73128b6c-5384-46cc-b6ea-5818a4014feb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:24.978387 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:24.987669 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f47be3-b99c-4cfb-9946-6ee434419399 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:24.994721 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f06be-e9f3-4239-ade3-15a668754ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:24.995803 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f06be-e9f3-4239-ade3-15a668754ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:24.996700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f06be-e9f3-4239-ade3-15a668754ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:24.997132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f06be-e9f3-4239-ade3-15a668754ff5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:25.006365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f06be-e9f3-4239-ade3-15a668754ff5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:25.008572 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f06be-e9f3-4239-ade3-15a668754ff5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:25.024122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8d6f06be-e9f3-4239-ade3-15a668754ff5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:25.024122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:25.024122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:25.024122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:25.024122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:25.024122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:25.024122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:25.024122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:25.024122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:25.024122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:25.024122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:25.024122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:25.024122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:25.024122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:25.024122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:25.025260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:25.025260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:25.025260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:25.025260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:25.025260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:25.025260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:25.025260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:25.025260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:25.025260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:25.025260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:25.025260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:25.025260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:25.025260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:25.025260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:25.025260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:25.025260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:25.025260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:25.026154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:25.026154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:25.026154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:25.026154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:25.026154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:25.026154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:25.026154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:25.026154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:25.030579 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5192ede-2368-467c-aa04-116f4c2bc89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:25.030995 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 408/803] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:25.031775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5192ede-2368-467c-aa04-116f4c2bc89a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:25.031902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f06be-e9f3-4239-ade3-15a668754ff5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:25.032434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5192ede-2368-467c-aa04-116f4c2bc89a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:25.032937 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5192ede-2368-467c-aa04-116f4c2bc89a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:25.046274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5192ede-2368-467c-aa04-116f4c2bc89a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:25.048713 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5192ede-2368-467c-aa04-116f4c2bc89a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:25.072874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a5192ede-2368-467c-aa04-116f4c2bc89a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:25.072874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:25.072874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:25.072874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:25.072874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:25.072874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:25.072874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:25.072874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:25.072874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:25.072874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:25.072874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:25.072874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:25.072874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:25.072874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:25.072874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:25.073825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:25.073825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:25.073825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:25.073825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:25.073825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:25.073825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:25.073825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:25.073825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:25.073825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:25.073825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:25.073825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:25.073825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:25.073825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:25.073825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:25.073825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:25.073825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:25.073825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:25.074742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:25.074742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:25.074742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:25.074742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:25.078804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 396/804] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:25.080119 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5192ede-2368-467c-aa04-116f4c2bc89a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:25.156558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f06be-e9f3-4239-ade3-15a668754ff5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:25.189779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5192ede-2368-467c-aa04-116f4c2bc89a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:25.191735 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:25.199571 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e5aca-2111-4b29-a728-fb774e2d9f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:25.200249 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e5aca-2111-4b29-a728-fb774e2d9f19 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:25.200558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e5aca-2111-4b29-a728-fb774e2d9f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:25.200843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e5aca-2111-4b29-a728-fb774e2d9f19 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:25.202042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e5aca-2111-4b29-a728-fb774e2d9f19 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:25.202351 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e5aca-2111-4b29-a728-fb774e2d9f19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:25.203727 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e5aca-2111-4b29-a728-fb774e2d9f19 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:25.215811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 409/805] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Jul 18 10:16:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:25.215811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e5aca-2111-4b29-a728-fb774e2d9f19 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:25.240592 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c1f9e-40d9-48c1-8226-7bc715d1fb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:25.241440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c1f9e-40d9-48c1-8226-7bc715d1fb3e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:25.241731 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c1f9e-40d9-48c1-8226-7bc715d1fb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:25.242013 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c1f9e-40d9-48c1-8226-7bc715d1fb3e None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:25.242811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c1f9e-40d9-48c1-8226-7bc715d1fb3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:25.243126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c1f9e-40d9-48c1-8226-7bc715d1fb3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:25.245126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c1f9e-40d9-48c1-8226-7bc715d1fb3e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:25.265669 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 397/806] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:25.265669 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c1f9e-40d9-48c1-8226-7bc715d1fb3e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:25.274156 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:25.339273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e5aca-2111-4b29-a728-fb774e2d9f19 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:25.376716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca245de-eb65-4f50-9af0-626c1de89a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:25.377384 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca245de-eb65-4f50-9af0-626c1de89a20 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:25.377668 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca245de-eb65-4f50-9af0-626c1de89a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:25.377944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca245de-eb65-4f50-9af0-626c1de89a20 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:25.378670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca245de-eb65-4f50-9af0-626c1de89a20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:25.378977 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca245de-eb65-4f50-9af0-626c1de89a20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:25.380341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca245de-eb65-4f50-9af0-626c1de89a20 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:25.383631 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c1f9e-40d9-48c1-8226-7bc715d1fb3e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:25.397939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 410/807] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:25.397939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca245de-eb65-4f50-9af0-626c1de89a20 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:25.427614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d8608-e0aa-4027-93a5-9943b968ebd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:25.428761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d8608-e0aa-4027-93a5-9943b968ebd4 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:25.429382 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d8608-e0aa-4027-93a5-9943b968ebd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:25.429891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d8608-e0aa-4027-93a5-9943b968ebd4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:25.430875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d8608-e0aa-4027-93a5-9943b968ebd4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:25.432482 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d8608-e0aa-4027-93a5-9943b968ebd4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:25.464810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 398/808] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jul 18 10:16:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:16:25.466251 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d8608-e0aa-4027-93a5-9943b968ebd4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:25.501407 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:25.530747 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca245de-eb65-4f50-9af0-626c1de89a20 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:25.543500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:25.592649 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d7c9e-c436-49bb-aece-569c4f6b228f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:25.593754 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d7c9e-c436-49bb-aece-569c4f6b228f None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:25.594143 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d7c9e-c436-49bb-aece-569c4f6b228f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:25.594521 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d7c9e-c436-49bb-aece-569c4f6b228f None admin] PATH_INFO: `/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/a09029144c454b099bea86aaa5229ef0/roles/f57b3e871ce142a090d6aa7dea9ac7d5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:25.612282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d8608-e0aa-4027-93a5-9943b968ebd4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:25.618354 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d7c9e-c436-49bb-aece-569c4f6b228f None admin] RBAC: Authorizing `identity:create_grant(project_id=e71a3acd0adf46a2a35b965a27c4998f, user_id=a09029144c454b099bea86aaa5229ef0, role_id=f57b3e871ce142a090d6aa7dea9ac7d5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:25.619992 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d7c9e-c436-49bb-aece-569c4f6b228f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:25.644966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 411/809] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:16:25 2022] PUT /identity/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/a09029144c454b099bea86aaa5229ef0/roles/f57b3e871ce142a090d6aa7dea9ac7d5 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:16:25.644966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf3ae9-f7dc-4170-90b1-3f162deb4de7 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:25.644966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf3ae9-f7dc-4170-90b1-3f162deb4de7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:25.644966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf3ae9-f7dc-4170-90b1-3f162deb4de7 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:25.647415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 412/810] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:25.649541 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437b21e-2ef8-4768-bbe1-030828cfc764 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:25.649839 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437b21e-2ef8-4768-bbe1-030828cfc764 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:25.650123 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437b21e-2ef8-4768-bbe1-030828cfc764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:25.672242 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a437b21e-2ef8-4768-bbe1-030828cfc764 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:25.673277 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a437b21e-2ef8-4768-bbe1-030828cfc764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:25.674000 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8eb0b-09a5-4841-86e4-ccc4a9d18267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:25.674836 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8eb0b-09a5-4841-86e4-ccc4a9d18267 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:25.675126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8eb0b-09a5-4841-86e4-ccc4a9d18267 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:25.675651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8eb0b-09a5-4841-86e4-ccc4a9d18267 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:25.686591 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8eb0b-09a5-4841-86e4-ccc4a9d18267 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:25.688085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8eb0b-09a5-4841-86e4-ccc4a9d18267 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:25.707300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-28b8eb0b-09a5-4841-86e4-ccc4a9d18267 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:25.707300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:25.707300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:25.707300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:25.707300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:25.707300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:25.707300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:25.707300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:25.707300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:25.707300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:25.707300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:25.707300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:25.707300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:25.707300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:25.707300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:25.708362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:25.708362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:25.708362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:25.708362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:25.708362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:25.708362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:25.708362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:25.708362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:25.708362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:25.708362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:25.708362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:25.708362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:25.708362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:25.708362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:25.708362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:25.708362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:25.708362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:25.709727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:25.709727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:25.709727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:25.709727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:25.709727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:25.709727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:25.709727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:25.709727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:25.709727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 399/811] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:25.710304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02126831-c1fd-499d-9d7c-f04e7a24cae3 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:25.715730 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02126831-c1fd-499d-9d7c-f04e7a24cae3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:25.715730 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02126831-c1fd-499d-9d7c-f04e7a24cae3 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:25.715730 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 400/812] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:25.718963 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02126831-c1fd-499d-9d7c-f04e7a24cae3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:25.776487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 413/813] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:25.778309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39daedbf-d92d-4a38-901a-e5b2e3a51100 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:25.778605 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39daedbf-d92d-4a38-901a-e5b2e3a51100 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:25.778884 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39daedbf-d92d-4a38-901a-e5b2e3a51100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:25.810235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-39daedbf-d92d-4a38-901a-e5b2e3a51100 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:25.812592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-39daedbf-d92d-4a38-901a-e5b2e3a51100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:25.847603 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:25.875109 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02126831-c1fd-499d-9d7c-f04e7a24cae3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:25.895672 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 414/814] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:25.900539 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39daedbf-d92d-4a38-901a-e5b2e3a51100 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:25.922464 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:25.936487 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158bb3b-665d-41fd-8b8f-75e9e60362f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:25.937854 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158bb3b-665d-41fd-8b8f-75e9e60362f1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:25.938785 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158bb3b-665d-41fd-8b8f-75e9e60362f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:25.939537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158bb3b-665d-41fd-8b8f-75e9e60362f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:25.940849 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158bb3b-665d-41fd-8b8f-75e9e60362f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:25.941735 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158bb3b-665d-41fd-8b8f-75e9e60362f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:25.944821 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158bb3b-665d-41fd-8b8f-75e9e60362f1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:25.961313 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 401/815] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:25.962680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158bb3b-665d-41fd-8b8f-75e9e60362f1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:26.026440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39daedbf-d92d-4a38-901a-e5b2e3a51100 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:26.072386 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878f139-eb9f-46aa-b87e-95dc371ff9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:26.073222 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878f139-eb9f-46aa-b87e-95dc371ff9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:26.073505 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878f139-eb9f-46aa-b87e-95dc371ff9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:26.073777 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878f139-eb9f-46aa-b87e-95dc371ff9a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:26.088175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878f139-eb9f-46aa-b87e-95dc371ff9a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:26.088785 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158bb3b-665d-41fd-8b8f-75e9e60362f1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:26.097465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-1878f139-eb9f-46aa-b87e-95dc371ff9a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:16:26.097465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:26.097465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:26.097465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:26.097465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:26.097465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:26.097465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:26.097465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:26.097465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:26.097465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:26.097465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:26.097465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:26.097465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:26.097465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:26.097465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:26.097465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:26.098462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:26.098462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:26.098462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:26.098462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:26.098462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:26.098462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:26.098462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:26.098462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:26.098462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:26.098462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:26.098462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:26.098462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:26.098462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:26.098462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:26.098462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:26.098462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:26.099296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:26.099296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:26.099296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:26.099296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:26.099296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:26.099296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:26.099296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:26.099296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:26.099296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:26.099296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:26.099296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:16:26.099296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:26.104408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 415/816] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:16:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:26.105563 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878f139-eb9f-46aa-b87e-95dc371ff9a5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:26.133486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7bb55-8534-436e-b4da-ff095e4e5ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:26.134753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7bb55-8534-436e-b4da-ff095e4e5ffd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:26.135244 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7bb55-8534-436e-b4da-ff095e4e5ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:26.135790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7bb55-8534-436e-b4da-ff095e4e5ffd None admin] PATH_INFO: `/v3/services/network` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:26.136717 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7bb55-8534-436e-b4da-ff095e4e5ffd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:26.138263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7bb55-8534-436e-b4da-ff095e4e5ffd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:26.153383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-1fe7bb55-8534-436e-b4da-ff095e4e5ffd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:16:26.153383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:26.153383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:26.153383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:26.153383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:26.153383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:26.153383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:26.153383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:26.153383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:16:26.153383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:26.153383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:26.153383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:26.153383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:26.153383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:26.153383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:26.153383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:26.153383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:26.154432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:26.154432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:26.154432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:26.154432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:26.154432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:26.154432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:26.154432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:26.154432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:26.154432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:26.154432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:26.154432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:26.154432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:26.154432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:26.154432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:26.154432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:26.154432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:26.155288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:26.155288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:26.155288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:26.155288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:26.155288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:26.155288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:26.155288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:26.155288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:26.155288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:26.155288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:26.155288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:26.155288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:26.155288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 10:16:26.155288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:26.161074 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 402/817] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:25 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:26.161074 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7bb55-8534-436e-b4da-ff095e4e5ffd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:26.164944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:26.194627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:26.228938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878f139-eb9f-46aa-b87e-95dc371ff9a5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:26.274795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a726c-3bdd-4953-8044-60835fb9c447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:26.276229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a726c-3bdd-4953-8044-60835fb9c447 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:26.277005 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a726c-3bdd-4953-8044-60835fb9c447 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:26.277447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a726c-3bdd-4953-8044-60835fb9c447 None admin] PATH_INFO: `/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/fd6168873f2846c8af73b903bba17eb8/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:26.280051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a726c-3bdd-4953-8044-60835fb9c447 None admin] RBAC: Authorizing `identity:create_grant(project_id=e71a3acd0adf46a2a35b965a27c4998f, user_id=fd6168873f2846c8af73b903bba17eb8, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:26.282458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a726c-3bdd-4953-8044-60835fb9c447 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:26.294077 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7bb55-8534-436e-b4da-ff095e4e5ffd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:26.300253 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 416/818] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:16:26 2022] PUT /identity/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/fd6168873f2846c8af73b903bba17eb8/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:16:26.301444 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a726c-3bdd-4953-8044-60835fb9c447 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:26.335789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbfcde-d0df-4175-b907-28cf6861b747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:26.336586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbfcde-d0df-4175-b907-28cf6861b747 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:26.336916 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbfcde-d0df-4175-b907-28cf6861b747 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:26.337197 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbfcde-d0df-4175-b907-28cf6861b747 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:26.338017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbfcde-d0df-4175-b907-28cf6861b747 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:26.338338 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbfcde-d0df-4175-b907-28cf6861b747 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:26.339882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbfcde-d0df-4175-b907-28cf6861b747 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:26.355011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 403/819] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:16:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:26.357578 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbfcde-d0df-4175-b907-28cf6861b747 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:26.431425 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a726c-3bdd-4953-8044-60835fb9c447 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:26.449849 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:26.466888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:26.473067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7cefdc-1ec6-4550-8640-ec4bcaf0e1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:26.473862 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cefdc-1ec6-4550-8640-ec4bcaf0e1ae None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:26.474676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cefdc-1ec6-4550-8640-ec4bcaf0e1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:26.475454 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7cefdc-1ec6-4550-8640-ec4bcaf0e1ae None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:26.477254 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7cefdc-1ec6-4550-8640-ec4bcaf0e1ae None admin] RBAC: Adding query filter params (name=network) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:26.477798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7cefdc-1ec6-4550-8640-ec4bcaf0e1ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:26.479370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7cefdc-1ec6-4550-8640-ec4bcaf0e1ae None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:26.486034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 417/820] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 18 10:16:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:26.492264 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7cefdc-1ec6-4550-8640-ec4bcaf0e1ae None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:26.511034 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbfcde-d0df-4175-b907-28cf6861b747 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:26.561704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52f0e7-5acb-4dc8-b77c-2e92b061a2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:26.562798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52f0e7-5acb-4dc8-b77c-2e92b061a2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:26.565696 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52f0e7-5acb-4dc8-b77c-2e92b061a2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:26.565696 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52f0e7-5acb-4dc8-b77c-2e92b061a2b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:26.588405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52f0e7-5acb-4dc8-b77c-2e92b061a2b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:26.594571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52f0e7-5acb-4dc8-b77c-2e92b061a2b1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:26.617597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8c52f0e7-5acb-4dc8-b77c-2e92b061a2b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:26.617597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:26.617597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:26.617597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:26.617597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:26.617597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:26.617597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:26.617597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:26.617597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:26.617597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:26.617597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:26.617597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:26.617597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:26.617597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:26.617597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:26.618934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:26.618934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:26.618934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:26.618934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:26.618934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:26.618934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:26.618934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:26.618934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:26.618934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:26.618934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:26.618934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:26.618934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:26.618934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:26.618934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:26.618934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:26.618934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:26.618934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:26.620227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:26.620227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:26.620227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:26.620227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:26.620227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 404/821] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:26.625236 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7cefdc-1ec6-4550-8640-ec4bcaf0e1ae None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:26.629151 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52f0e7-5acb-4dc8-b77c-2e92b061a2b1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:26.661189 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50f9bd-93c6-4006-91b6-b08c11e0b3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:26.662030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50f9bd-93c6-4006-91b6-b08c11e0b3df None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:26.662437 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50f9bd-93c6-4006-91b6-b08c11e0b3df None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:26.663354 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50f9bd-93c6-4006-91b6-b08c11e0b3df None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:26.664231 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50f9bd-93c6-4006-91b6-b08c11e0b3df None admin] RBAC: Adding query filter params (type=network) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:26.664685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50f9bd-93c6-4006-91b6-b08c11e0b3df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:26.666166 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50f9bd-93c6-4006-91b6-b08c11e0b3df None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:26.671620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 418/822] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 18 10:16:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:26.677102 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50f9bd-93c6-4006-91b6-b08c11e0b3df None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:26.750122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:26.756120 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52f0e7-5acb-4dc8-b77c-2e92b061a2b1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:26.774627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:26.797305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50f9bd-93c6-4006-91b6-b08c11e0b3df None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:26.797970 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c800328-7f51-406e-8a47-48802f51d708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:26.799014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c800328-7f51-406e-8a47-48802f51d708 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:26.799637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c800328-7f51-406e-8a47-48802f51d708 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:26.800147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c800328-7f51-406e-8a47-48802f51d708 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:26.801125 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c800328-7f51-406e-8a47-48802f51d708 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:26.801642 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c800328-7f51-406e-8a47-48802f51d708 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:26.803215 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c800328-7f51-406e-8a47-48802f51d708 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:26.817727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 405/823] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:26.825846 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c800328-7f51-406e-8a47-48802f51d708 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:26.847827 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd1ab0-af11-4d22-b542-d2cbf971367d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:26.849981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd1ab0-af11-4d22-b542-d2cbf971367d None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:26.851023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd1ab0-af11-4d22-b542-d2cbf971367d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:26.852136 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd1ab0-af11-4d22-b542-d2cbf971367d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:26.853137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd1ab0-af11-4d22-b542-d2cbf971367d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:26.856660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd1ab0-af11-4d22-b542-d2cbf971367d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:26.894446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 419/824] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jul 18 10:16:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:16:26.981718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c800328-7f51-406e-8a47-48802f51d708 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:27.041402 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb7be3-8bde-4b6f-ae12-cfe1cd46d488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:27.042543 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb7be3-8bde-4b6f-ae12-cfe1cd46d488 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:27.042981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb7be3-8bde-4b6f-ae12-cfe1cd46d488 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:27.043412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb7be3-8bde-4b6f-ae12-cfe1cd46d488 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:27.055376 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb7be3-8bde-4b6f-ae12-cfe1cd46d488 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:27.057147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb7be3-8bde-4b6f-ae12-cfe1cd46d488 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:27.077734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-15bb7be3-8bde-4b6f-ae12-cfe1cd46d488 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:16:27.077734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:27.077734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:27.077734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:27.077734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:27.077734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:27.077734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:27.077734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:27.077734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:27.077734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:27.077734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:27.077734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:27.077734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:27.077734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:27.077734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:27.077734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:27.078939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:27.078939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:27.078939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:27.078939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:27.078939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:27.078939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:27.078939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:27.078939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:27.078939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:27.078939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:27.078939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:27.078939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:27.078939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:27.078939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:27.078939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:27.078939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:27.078939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:27.078939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:27.080176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:27.080176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:27.080176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:27.080176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:27.080176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:27.080176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:27.080176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:27.080176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:27.080176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:16:27.080176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:27.082189 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 406/825] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jul 18 10:16:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:27.088168 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd1ab0-af11-4d22-b542-d2cbf971367d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:27.240113 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd1ab0-af11-4d22-b542-d2cbf971367d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:27.262396 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:27.283920 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83460b23-193c-429e-afb1-26a1662ea0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:27.285000 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83460b23-193c-429e-afb1-26a1662ea0fd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:27.285439 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83460b23-193c-429e-afb1-26a1662ea0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:27.285897 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83460b23-193c-429e-afb1-26a1662ea0fd None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:27.286892 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83460b23-193c-429e-afb1-26a1662ea0fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:27.287356 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83460b23-193c-429e-afb1-26a1662ea0fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:27.289561 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83460b23-193c-429e-afb1-26a1662ea0fd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:27.310512 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 420/826] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jul 18 10:16:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:27.315825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb7be3-8bde-4b6f-ae12-cfe1cd46d488 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:27.356587 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:27.440380 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb7be3-8bde-4b6f-ae12-cfe1cd46d488 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:27.483938 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50684b74-97d6-4f7c-a74a-d83926d92034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:27.484968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50684b74-97d6-4f7c-a74a-d83926d92034 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:27.485409 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50684b74-97d6-4f7c-a74a-d83926d92034 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:27.485866 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50684b74-97d6-4f7c-a74a-d83926d92034 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:27.563880 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50684b74-97d6-4f7c-a74a-d83926d92034 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:27.565709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50684b74-97d6-4f7c-a74a-d83926d92034 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:27.597700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:27.675818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 407/827] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:16:27 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:16:27.681279 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83460b23-193c-429e-afb1-26a1662ea0fd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:27.785825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83460b23-193c-429e-afb1-26a1662ea0fd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:27.812640 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5558c-9ca5-49fa-a1b2-a6477450d6b4 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:27.812965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5558c-9ca5-49fa-a1b2-a6477450d6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:27.813219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5558c-9ca5-49fa-a1b2-a6477450d6b4 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:27.814814 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 408/828] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:27.820016 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1b215-a66f-4082-8d15-556d51af0bba None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:27.820310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1b215-a66f-4082-8d15-556d51af0bba None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:27.820718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d1b215-a66f-4082-8d15-556d51af0bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:27.822887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59f84a-6764-4dec-bf9e-025625af649c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:27.823676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59f84a-6764-4dec-bf9e-025625af649c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:27.823957 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59f84a-6764-4dec-bf9e-025625af649c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:27.824233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59f84a-6764-4dec-bf9e-025625af649c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:27.835565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59f84a-6764-4dec-bf9e-025625af649c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:27.837045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59f84a-6764-4dec-bf9e-025625af649c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:27.839086 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-81d1b215-a66f-4082-8d15-556d51af0bba None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:27.840537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-81d1b215-a66f-4082-8d15-556d51af0bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:27.855851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-1c59f84a-6764-4dec-bf9e-025625af649c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:27.855851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:27.855851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:27.855851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:27.855851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:27.855851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:27.855851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:27.855851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:27.855851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:27.855851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:27.855851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:27.855851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:27.855851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:27.855851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:27.855851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:27.856833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:27.856833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:27.856833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:27.856833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:27.856833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:27.856833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:27.856833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:27.856833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:27.856833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:27.856833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:27.856833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:27.856833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:27.856833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:27.856833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:27.856833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:27.856833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:27.856833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:27.857741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:27.857741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:27.857741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:27.857741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:27.861718 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 421/829] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:27.862996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59f84a-6764-4dec-bf9e-025625af649c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:27.892747 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:27.918384 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 409/830] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:16:27.940499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1b215-a66f-4082-8d15-556d51af0bba None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:27.971729 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59f84a-6764-4dec-bf9e-025625af649c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:28.008631 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:28.009128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db916f-d354-45b2-bae4-21d726307d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:28.009587 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db916f-d354-45b2-bae4-21d726307d1e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:28.009587 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db916f-d354-45b2-bae4-21d726307d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:28.009882 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db916f-d354-45b2-bae4-21d726307d1e None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:28.010945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db916f-d354-45b2-bae4-21d726307d1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:28.010945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db916f-d354-45b2-bae4-21d726307d1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:28.012736 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db916f-d354-45b2-bae4-21d726307d1e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:28.028732 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 422/831] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:28.028732 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a854a-9da5-469c-94ce-8353e9b27401 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:28.028732 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a854a-9da5-469c-94ce-8353e9b27401 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:28.028732 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a854a-9da5-469c-94ce-8353e9b27401 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:28.028732 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 423/832] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:28.032201 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a854a-9da5-469c-94ce-8353e9b27401 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:28.054592 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d1b215-a66f-4082-8d15-556d51af0bba None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:28.093205 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed7659-d0cf-4883-a070-da5ffb39f4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:28.094443 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed7659-d0cf-4883-a070-da5ffb39f4af None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:28.094874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed7659-d0cf-4883-a070-da5ffb39f4af None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:28.095157 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed7659-d0cf-4883-a070-da5ffb39f4af None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:28.095817 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed7659-d0cf-4883-a070-da5ffb39f4af None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:28.097331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed7659-d0cf-4883-a070-da5ffb39f4af None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:28.110287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-13ed7659-d0cf-4883-a070-da5ffb39f4af None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:16:28.110287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:28.110287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:28.110287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:28.110287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:28.110287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:28.110287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:28.110287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:28.110287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:16:28.110287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:28.110287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:28.110287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:28.110287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:28.110287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:28.110287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:28.110287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:28.110287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:28.111326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:28.111326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:28.111326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:28.111326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:28.111326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:28.111326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:28.111326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:28.111326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:28.111326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:28.111326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:28.111326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:28.111326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:28.111326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:28.111326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:28.111326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:28.111326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:28.112176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:28.112176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:28.112176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:28.112176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:28.112176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:28.112176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:28.112176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:28.112176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:28.112176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:28.112176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:28.112176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:28.112176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:28.112176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:16:28.112176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:28.112176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 410/833] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:16:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:28.114019 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b736669-876f-4c28-8a34-1e1a60b26ae4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:28.114019 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b736669-876f-4c28-8a34-1e1a60b26ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:28.114019 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b736669-876f-4c28-8a34-1e1a60b26ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:28.128160 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:28.130200 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8b736669-876f-4c28-8a34-1e1a60b26ae4 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:28.131441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8b736669-876f-4c28-8a34-1e1a60b26ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:28.149651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a854a-9da5-469c-94ce-8353e9b27401 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:28.188350 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3a083-4ca7-4821-916d-e979a07f9dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:28.189176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3a083-4ca7-4821-916d-e979a07f9dfe None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:28.189464 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3a083-4ca7-4821-916d-e979a07f9dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:28.189741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3a083-4ca7-4821-916d-e979a07f9dfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:28.197175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3a083-4ca7-4821-916d-e979a07f9dfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:28.198573 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3a083-4ca7-4821-916d-e979a07f9dfe None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:28.205751 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 411/834] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:28.206768 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b736669-876f-4c28-8a34-1e1a60b26ae4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:28.213173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-86e3a083-4ca7-4821-916d-e979a07f9dfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:28.213173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:28.213173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:28.213173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:28.213173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:28.213173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:28.213173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:28.213173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:28.213173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:28.213173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:28.213173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:28.213173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:28.213173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:28.213173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:28.213173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:28.214068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:28.214068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:28.214068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:28.214068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:28.214068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:28.214068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:28.214068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:28.214068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:28.214068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:28.214068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:28.214068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:28.214068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:28.214068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:28.214068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:28.214068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:28.214068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:28.214068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:28.214982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:28.214982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:28.214982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:28.214982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:28.214982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:28.214982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:28.214982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:28.214982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:28.214982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 424/835] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:28.221690 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3a083-4ca7-4821-916d-e979a07f9dfe None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:28.274347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:28.324551 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b736669-876f-4c28-8a34-1e1a60b26ae4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:28.326941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3a083-4ca7-4821-916d-e979a07f9dfe None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:28.364306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e88365-d8ea-4d4e-816d-321a240c615f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:28.365017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e88365-d8ea-4d4e-816d-321a240c615f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:28.365306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e88365-d8ea-4d4e-816d-321a240c615f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:28.365589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e88365-d8ea-4d4e-816d-321a240c615f None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:28.366421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e88365-d8ea-4d4e-816d-321a240c615f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:28.366740 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e88365-d8ea-4d4e-816d-321a240c615f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:28.367412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4474ab-2650-48ae-a17d-755b22efabb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:28.368142 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4474ab-2650-48ae-a17d-755b22efabb1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:28.368235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e88365-d8ea-4d4e-816d-321a240c615f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:28.368535 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4474ab-2650-48ae-a17d-755b22efabb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:28.368816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4474ab-2650-48ae-a17d-755b22efabb1 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:28.369526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4474ab-2650-48ae-a17d-755b22efabb1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:28.369833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4474ab-2650-48ae-a17d-755b22efabb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:28.371176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4474ab-2650-48ae-a17d-755b22efabb1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:28.380488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 412/836] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:16:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:28.380488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4474ab-2650-48ae-a17d-755b22efabb1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:28.385776 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 425/837] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:28.386783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e88365-d8ea-4d4e-816d-321a240c615f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:28.396069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:28.493309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e88365-d8ea-4d4e-816d-321a240c615f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:28.495057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4474ab-2650-48ae-a17d-755b22efabb1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:28.515629 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:28.530697 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c13777-c09b-47dc-aa38-7c1b0580becb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:28.531020 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb0bac-e7e6-4a69-bde9-5fe546703a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:28.531368 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c13777-c09b-47dc-aa38-7c1b0580becb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:28.531658 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c13777-c09b-47dc-aa38-7c1b0580becb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:28.531814 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb0bac-e7e6-4a69-bde9-5fe546703a41 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:28.531979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c13777-c09b-47dc-aa38-7c1b0580becb None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:28.532126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb0bac-e7e6-4a69-bde9-5fe546703a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:28.532370 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb0bac-e7e6-4a69-bde9-5fe546703a41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:28.532697 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c13777-c09b-47dc-aa38-7c1b0580becb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:28.533010 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c13777-c09b-47dc-aa38-7c1b0580becb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:28.534399 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c13777-c09b-47dc-aa38-7c1b0580becb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:28.548886 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 426/838] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:16:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:28.548886 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c13777-c09b-47dc-aa38-7c1b0580becb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:28.549174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb0bac-e7e6-4a69-bde9-5fe546703a41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:28.551649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-21eb0bac-e7e6-4a69-bde9-5fe546703a41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:28.551649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:28.551649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:28.551649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:28.551649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:28.551649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:28.551649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:28.551649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:28.551649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:28.551649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:28.551649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:28.551649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:28.551649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:28.551649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:28.551649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:28.551649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:28.552680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:28.552680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:28.552680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:28.552680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:28.552680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:28.552680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:28.552680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:28.552680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:28.552680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:28.552680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:28.552680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:28.552680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:28.552680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:28.552680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:28.552680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:28.552680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:28.553589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:28.553589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:28.553589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:28.553589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:28.553589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:28.553589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:28.553589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:28.553589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:28.553589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:28.553589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:28.553589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:16:28.553589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:28.553589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 413/839] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:16:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:28.563262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb0bac-e7e6-4a69-bde9-5fe546703a41 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:28.638661 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:28.674815 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c13777-c09b-47dc-aa38-7c1b0580becb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:28.716260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb0bac-e7e6-4a69-bde9-5fe546703a41 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:28.725151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6e8c2-62f7-41cc-b941-e51610e21f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:28.725922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6e8c2-62f7-41cc-b941-e51610e21f53 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:28.726205 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6e8c2-62f7-41cc-b941-e51610e21f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:28.726484 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6e8c2-62f7-41cc-b941-e51610e21f53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:28.727774 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6e8c2-62f7-41cc-b941-e51610e21f53 None admin] RBAC: Adding query filter params (role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=29048c2021e340b081ae44a390583e7a) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:28.728088 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6e8c2-62f7-41cc-b941-e51610e21f53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:28.730474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6e8c2-62f7-41cc-b941-e51610e21f53 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:28.738962 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 427/840] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:16:28 2022] GET /identity/v3/role_assignments?user.id=29048c2021e340b081ae44a390583e7a&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:28.756306 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e36fa-7c82-4488-b601-fd0b71606be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:28.757602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e36fa-7c82-4488-b601-fd0b71606be6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:28.757957 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e36fa-7c82-4488-b601-fd0b71606be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:28.758309 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e36fa-7c82-4488-b601-fd0b71606be6 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:28.759872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e36fa-7c82-4488-b601-fd0b71606be6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:28.759872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e36fa-7c82-4488-b601-fd0b71606be6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:28.760970 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e36fa-7c82-4488-b601-fd0b71606be6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:28.776764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 414/841] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:16:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:28.780103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6e8c2-62f7-41cc-b941-e51610e21f53 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:28.833773 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b69e8f-cbd7-4cb3-80ea-e9d65eb75c55 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:28.834862 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b69e8f-cbd7-4cb3-80ea-e9d65eb75c55 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:28.835648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b69e8f-cbd7-4cb3-80ea-e9d65eb75c55 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:28.837920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 415/842] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:28.844929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16e65c-5172-42c3-a393-28cf50df833d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:28.845234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16e65c-5172-42c3-a393-28cf50df833d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:28.845507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16e65c-5172-42c3-a393-28cf50df833d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:28.867410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ee16e65c-5172-42c3-a393-28cf50df833d None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:28.869647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ee16e65c-5172-42c3-a393-28cf50df833d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:28.906417 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6e8c2-62f7-41cc-b941-e51610e21f53 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:28.918222 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:28.951084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7b863-345c-454d-9326-eb67198edf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:28.951897 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7b863-345c-454d-9326-eb67198edf95 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:28.952357 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7b863-345c-454d-9326-eb67198edf95 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:28.952670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7b863-345c-454d-9326-eb67198edf95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:28.955335 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:28.964527 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 416/843] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:16:28.964952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7b863-345c-454d-9326-eb67198edf95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:28.966419 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7b863-345c-454d-9326-eb67198edf95 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:28.982235 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16e65c-5172-42c3-a393-28cf50df833d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:28.987552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b2f7b863-345c-454d-9326-eb67198edf95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:28.987552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:28.987552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:28.987552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:28.987552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:28.987552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:28.987552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:28.987552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:28.987552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:28.987552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:28.987552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:28.987552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:28.987552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:28.987552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:28.987552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:28.988461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:28.988461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:28.988461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:28.988461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:28.988461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:28.988461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:28.988461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:28.988461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:28.988461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:28.988461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:28.988461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:28.988461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:28.988461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:28.988461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:28.988461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:28.988461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:28.988461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:28.989398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:28.989398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:28.989398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:28.989398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:28.989398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 428/844] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:28.995421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7b863-345c-454d-9326-eb67198edf95 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:29.110734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16e65c-5172-42c3-a393-28cf50df833d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:29.118341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7b863-345c-454d-9326-eb67198edf95 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:29.150307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e683a28-e5c1-4e60-ad99-b9c2fb28da38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:29.150974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e683a28-e5c1-4e60-ad99-b9c2fb28da38 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:29.151267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e683a28-e5c1-4e60-ad99-b9c2fb28da38 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:29.151548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e683a28-e5c1-4e60-ad99-b9c2fb28da38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:29.156133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1972cd5-5511-4360-a91a-7d0cc5f44c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:29.157115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1972cd5-5511-4360-a91a-7d0cc5f44c3b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:29.157610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1972cd5-5511-4360-a91a-7d0cc5f44c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:29.158276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1972cd5-5511-4360-a91a-7d0cc5f44c3b None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:29.159212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1972cd5-5511-4360-a91a-7d0cc5f44c3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:29.159741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1972cd5-5511-4360-a91a-7d0cc5f44c3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:29.161376 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1972cd5-5511-4360-a91a-7d0cc5f44c3b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:29.163263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e683a28-e5c1-4e60-ad99-b9c2fb28da38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:29.171327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0e683a28-e5c1-4e60-ad99-b9c2fb28da38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:16:29.171327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:29.171327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:29.171327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:29.171327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:29.171327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:29.171327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:29.171327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:29.171327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:29.171327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:29.171327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:29.171327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:29.171327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:29.171327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:29.171327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:29.171327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:29.172291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:29.172291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:29.172291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:29.172291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:29.172291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:29.172291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:29.172291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:29.172291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:29.172291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:29.172291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:29.172291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:29.172291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:29.172291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:29.172291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:29.172291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:29.172291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:29.173171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:29.173171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:29.173171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:29.173171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:29.173171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:29.173171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:29.173171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:29.173171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:29.173171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:29.173171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:29.173171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:16:29.173171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:29.173171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 417/845] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:16:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:29.184628 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e683a28-e5c1-4e60-ad99-b9c2fb28da38 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:29.184810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 429/846] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:29.188239 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1972cd5-5511-4360-a91a-7d0cc5f44c3b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:29.220828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:29.254907 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:29.311489 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e683a28-e5c1-4e60-ad99-b9c2fb28da38 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:29.312872 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1972cd5-5511-4360-a91a-7d0cc5f44c3b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:29.354292 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c425cf7-bfcf-417a-8448-d24263f7fe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:29.355639 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c425cf7-bfcf-417a-8448-d24263f7fe47 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:29.356077 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c425cf7-bfcf-417a-8448-d24263f7fe47 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:29.356550 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c425cf7-bfcf-417a-8448-d24263f7fe47 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:29.357389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce1fc9-84de-4afb-ad9d-10fb32f4e782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:29.357717 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c425cf7-bfcf-417a-8448-d24263f7fe47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:29.358179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c425cf7-bfcf-417a-8448-d24263f7fe47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:29.358304 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce1fc9-84de-4afb-ad9d-10fb32f4e782 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:29.358466 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce1fc9-84de-4afb-ad9d-10fb32f4e782 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:29.358741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce1fc9-84de-4afb-ad9d-10fb32f4e782 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:29.360274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c425cf7-bfcf-417a-8448-d24263f7fe47 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:29.367228 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce1fc9-84de-4afb-ad9d-10fb32f4e782 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:29.369524 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce1fc9-84de-4afb-ad9d-10fb32f4e782 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:29.371928 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 418/847] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:16:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:29.377501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c425cf7-bfcf-417a-8448-d24263f7fe47 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:29.387762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-36ce1fc9-84de-4afb-ad9d-10fb32f4e782 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:16:29.387762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:29.387762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:29.387762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:29.387762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:29.387762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:29.387762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:29.387762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:29.387762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:29.387762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:29.387762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:29.387762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:29.387762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:29.387762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:29.387762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:29.387762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:29.389011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:29.389011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:29.389011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:29.389011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:29.389011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:29.389011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:29.389011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:29.389011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:29.389011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:29.389011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:29.389011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:29.389011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:29.389011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:29.389011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:29.389011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:29.389011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:29.389011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:29.389011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:29.390035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:29.390035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:29.390035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:29.390035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:29.390035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:29.390035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:29.390035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:29.390035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:29.390035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:16:29.390035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:29.392355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 430/848] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 18 10:16:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:29.397597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce1fc9-84de-4afb-ad9d-10fb32f4e782 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:29.478312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:29.486488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c425cf7-bfcf-417a-8448-d24263f7fe47 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:29.507849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce1fc9-84de-4afb-ad9d-10fb32f4e782 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:29.523304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c2619-49bb-4377-89a5-113652b94592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:29.524190 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c2619-49bb-4377-89a5-113652b94592 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:29.524804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c2619-49bb-4377-89a5-113652b94592 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:29.524939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c2619-49bb-4377-89a5-113652b94592 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:29.525034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:29.540182 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c2619-49bb-4377-89a5-113652b94592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:29.542220 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c2619-49bb-4377-89a5-113652b94592 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:29.546730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f03860b-b94b-4903-8274-d2cbacc27485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:29.547663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03860b-b94b-4903-8274-d2cbacc27485 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:29.548151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03860b-b94b-4903-8274-d2cbacc27485 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:29.548691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03860b-b94b-4903-8274-d2cbacc27485 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:29.549841 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f03860b-b94b-4903-8274-d2cbacc27485 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:29.550378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f03860b-b94b-4903-8274-d2cbacc27485 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:29.552195 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f03860b-b94b-4903-8274-d2cbacc27485 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:29.562582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8f2c2619-49bb-4377-89a5-113652b94592 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:29.562582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:29.562582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:29.562582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:29.562582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:29.562582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:29.562582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:29.562582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:29.562582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:29.562582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:29.562582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:29.562582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:29.562582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:29.562582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:29.562582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:29.563495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:29.563495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:29.563495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:29.563495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:29.563495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:29.563495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:29.563495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:29.563495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:29.563495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:29.563495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:29.563495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:29.563495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:29.563495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:29.563495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:29.563495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:29.563495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:29.563495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:29.564384 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:29.564384 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:29.564384 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:29.564384 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:29.568589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 419/849] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:29.569983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c2619-49bb-4377-89a5-113652b94592 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:29.573919 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 431/850] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Jul 18 10:16:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:29.580932 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f03860b-b94b-4903-8274-d2cbacc27485 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:29.692660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f03860b-b94b-4903-8274-d2cbacc27485 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:29.694502 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c2619-49bb-4377-89a5-113652b94592 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:29.733438 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c8e80-c80b-4345-b7c2-e9cea602859a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:29.734086 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c8e80-c80b-4345-b7c2-e9cea602859a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:29.734417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c8e80-c80b-4345-b7c2-e9cea602859a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:29.734650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c8e80-c80b-4345-b7c2-e9cea602859a None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:29.735586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c8e80-c80b-4345-b7c2-e9cea602859a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:29.735900 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c8e80-c80b-4345-b7c2-e9cea602859a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:29.736532 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2ac34-a14c-420f-9841-a4a75becfd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:29.737316 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c8e80-c80b-4345-b7c2-e9cea602859a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:29.737435 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2ac34-a14c-420f-9841-a4a75becfd2e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:29.737686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2ac34-a14c-420f-9841-a4a75becfd2e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:29.738048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2ac34-a14c-420f-9841-a4a75becfd2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:29.750822 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 420/851] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:29.756067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c8e80-c80b-4345-b7c2-e9cea602859a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:29.765726 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:29.777488 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:29.826805 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2ac34-a14c-420f-9841-a4a75becfd2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:29.828839 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2ac34-a14c-420f-9841-a4a75becfd2e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:29.872008 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c8e80-c80b-4345-b7c2-e9cea602859a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:29.910150 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d7195-852d-403a-b51e-0df462461f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:29.910947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d7195-852d-403a-b51e-0df462461f70 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:29.911234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d7195-852d-403a-b51e-0df462461f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:29.911519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d7195-852d-403a-b51e-0df462461f70 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:29.920757 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d7195-852d-403a-b51e-0df462461f70 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:29.922206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d7195-852d-403a-b51e-0df462461f70 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:29.938477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-247d7195-852d-403a-b51e-0df462461f70 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:16:29.938477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:29.938477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:29.938477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:29.938477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:29.938477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:29.938477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:29.938477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:29.938477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:29.938477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:29.938477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:29.938477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:29.938477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:29.938477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:29.938477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:29.938477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:29.940604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:29.940604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:29.940604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:29.940604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:29.940604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:29.940604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:29.940604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:29.940604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:29.940604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:29.940604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:29.940604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:29.940604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:29.940604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:29.940604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:29.940604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:29.940604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:29.940604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:29.940604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:29.942994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:29.942994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:29.942994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:29.942994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:29.942994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:29.942994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:29.942994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:29.942994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:29.942994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 10:16:29.942994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:29.948608 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 421/852] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:16:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:29.948608 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d7195-852d-403a-b51e-0df462461f70 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:29.958655 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 432/853] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:16:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 10:16:29.961629 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2ac34-a14c-420f-9841-a4a75becfd2e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:30.030110 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:30.035479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:30.075056 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d7195-852d-403a-b51e-0df462461f70 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:30.082335 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2ac34-a14c-420f-9841-a4a75becfd2e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:30.113053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe21fe-f36a-48c7-8a5d-aa434fe0cef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:30.114427 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe21fe-f36a-48c7-8a5d-aa434fe0cef9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:30.114917 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe21fe-f36a-48c7-8a5d-aa434fe0cef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:30.115394 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe21fe-f36a-48c7-8a5d-aa434fe0cef9 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:30.116482 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe21fe-f36a-48c7-8a5d-aa434fe0cef9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:30.117015 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe21fe-f36a-48c7-8a5d-aa434fe0cef9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:30.118684 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe21fe-f36a-48c7-8a5d-aa434fe0cef9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:30.126598 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567539ed-efa1-452d-bcff-ec282f7191f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:30.127406 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567539ed-efa1-452d-bcff-ec282f7191f0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:30.127686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567539ed-efa1-452d-bcff-ec282f7191f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:30.127960 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567539ed-efa1-452d-bcff-ec282f7191f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:30.143364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567539ed-efa1-452d-bcff-ec282f7191f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:30.143364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567539ed-efa1-452d-bcff-ec282f7191f0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:30.143948 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 422/854] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Jul 18 10:16:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:30.145107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe21fe-f36a-48c7-8a5d-aa434fe0cef9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:30.169487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-567539ed-efa1-452d-bcff-ec282f7191f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:30.169487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:30.169487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:30.169487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:30.169487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:30.169487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:30.169487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:30.169487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:30.169487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:30.169487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:30.169487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:30.169487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:30.169487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:30.169487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:30.169487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:30.170388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:30.170388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:30.170388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:30.170388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:30.170388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:30.170388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:30.170388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:30.170388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:30.170388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:30.170388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:30.170388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:30.170388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:30.170388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:30.170388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:30.170388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:30.170388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:30.170388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:30.171302 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:30.171302 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:30.171302 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:30.171302 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:30.177531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 433/855] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:30.177531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567539ed-efa1-452d-bcff-ec282f7191f0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:30.267297 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe21fe-f36a-48c7-8a5d-aa434fe0cef9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:30.300390 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:30.301361 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567539ed-efa1-452d-bcff-ec282f7191f0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:30.317232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce098558-7787-47bf-b897-44f461e5c3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:30.317882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce098558-7787-47bf-b897-44f461e5c3f3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:30.318168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce098558-7787-47bf-b897-44f461e5c3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:30.318446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce098558-7787-47bf-b897-44f461e5c3f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:30.323244 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:30.351706 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f90cff-088f-4bb7-8be2-1cc3bde53d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:30.352368 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f90cff-088f-4bb7-8be2-1cc3bde53d8b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:30.352696 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f90cff-088f-4bb7-8be2-1cc3bde53d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:30.352979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f90cff-088f-4bb7-8be2-1cc3bde53d8b None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:30.353970 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f90cff-088f-4bb7-8be2-1cc3bde53d8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:30.354276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f90cff-088f-4bb7-8be2-1cc3bde53d8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:30.356348 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f90cff-088f-4bb7-8be2-1cc3bde53d8b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:30.378966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 434/856] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:30.380818 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f90cff-088f-4bb7-8be2-1cc3bde53d8b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:30.404855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce098558-7787-47bf-b897-44f461e5c3f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:30.406602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce098558-7787-47bf-b897-44f461e5c3f3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:30.511643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f90cff-088f-4bb7-8be2-1cc3bde53d8b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:30.523599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 423/857] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:16:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:16:30.524822 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce098558-7787-47bf-b897-44f461e5c3f3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:30.549917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ccb45-5f39-4f1d-9036-4bbcfd61538c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:30.551061 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ccb45-5f39-4f1d-9036-4bbcfd61538c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:30.551548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ccb45-5f39-4f1d-9036-4bbcfd61538c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:30.552039 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ccb45-5f39-4f1d-9036-4bbcfd61538c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:30.554838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:30.560730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ccb45-5f39-4f1d-9036-4bbcfd61538c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:30.562565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ccb45-5f39-4f1d-9036-4bbcfd61538c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:30.577930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-bc3ccb45-5f39-4f1d-9036-4bbcfd61538c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:30.577930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:30.577930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:30.577930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:30.577930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:30.577930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:30.577930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:30.577930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:30.577930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:30.577930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:30.577930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:30.577930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:30.577930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:30.577930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:30.577930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:30.578916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:30.578916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:30.578916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:30.578916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:30.578916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:30.578916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:30.578916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:30.578916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:30.578916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:30.578916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:30.578916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:30.578916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:30.578916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:30.578916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:30.578916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:30.578916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:30.578916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:30.579826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:30.579826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:30.579826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:30.579826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:30.579826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:30.579826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:30.579826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:30.579826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:30.582324 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 435/858] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:30.590893 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ccb45-5f39-4f1d-9036-4bbcfd61538c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:30.627449 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:30.643922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce098558-7787-47bf-b897-44f461e5c3f3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:30.680998 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0f86a-767f-4e7a-8aea-d98b3325dc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:30.682193 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0f86a-767f-4e7a-8aea-d98b3325dc64 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:30.682682 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0f86a-767f-4e7a-8aea-d98b3325dc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:30.683178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0f86a-767f-4e7a-8aea-d98b3325dc64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:30.694979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0f86a-767f-4e7a-8aea-d98b3325dc64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:30.696558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ccb45-5f39-4f1d-9036-4bbcfd61538c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:30.696994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0f86a-767f-4e7a-8aea-d98b3325dc64 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:30.716664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-5ba0f86a-767f-4e7a-8aea-d98b3325dc64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:30.716664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:30.716664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:30.716664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:30.716664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:30.716664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:30.716664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:30.716664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:30.716664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:30.716664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:30.716664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:30.716664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:30.716664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:30.716664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:30.716664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:30.717759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:30.717759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:30.717759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:30.717759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:30.717759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:30.717759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:30.717759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:30.717759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:30.717759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:30.717759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:30.717759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:30.717759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:30.717759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:30.717759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:30.717759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:30.717759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:30.717759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:30.718820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:30.718820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:30.718820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:30.718820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:30.718820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 424/859] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:30.720491 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193b523-fee4-40b0-9b7c-2ce2ef521c35 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:30.721010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193b523-fee4-40b0-9b7c-2ce2ef521c35 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:30.721477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193b523-fee4-40b0-9b7c-2ce2ef521c35 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:30.723975 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 425/860] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:30.728398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193b523-fee4-40b0-9b7c-2ce2ef521c35 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:30.742737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ee0e5-be3e-461d-a659-022313bcff57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:30.743395 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ee0e5-be3e-461d-a659-022313bcff57 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:30.743678 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ee0e5-be3e-461d-a659-022313bcff57 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:30.743954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ee0e5-be3e-461d-a659-022313bcff57 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:30.744770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ee0e5-be3e-461d-a659-022313bcff57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:30.745078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ee0e5-be3e-461d-a659-022313bcff57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:30.746507 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ee0e5-be3e-461d-a659-022313bcff57 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:30.758791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 436/861] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:30.764839 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffef35a-4c84-440d-b71a-97d3028cc98c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:30.765131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffef35a-4c84-440d-b71a-97d3028cc98c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:30.765409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffef35a-4c84-440d-b71a-97d3028cc98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:30.783333 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bffef35a-4c84-440d-b71a-97d3028cc98c None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:30.784755 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bffef35a-4c84-440d-b71a-97d3028cc98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:30.813440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:30.842992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193b523-fee4-40b0-9b7c-2ce2ef521c35 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:30.864574 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 437/862] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:30.864574 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffef35a-4c84-440d-b71a-97d3028cc98c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:30.886373 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:30.890095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6490d-d60e-4023-a816-a5e185f1e670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:30.890795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6490d-d60e-4023-a816-a5e185f1e670 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:30.891083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6490d-d60e-4023-a816-a5e185f1e670 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:30.891373 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6490d-d60e-4023-a816-a5e185f1e670 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:30.892119 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6490d-d60e-4023-a816-a5e185f1e670 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:30.892383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6490d-d60e-4023-a816-a5e185f1e670 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:30.893775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6490d-d60e-4023-a816-a5e185f1e670 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:30.905637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 426/863] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:30.906426 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6490d-d60e-4023-a816-a5e185f1e670 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:30.960202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffef35a-4c84-440d-b71a-97d3028cc98c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:30.993406 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6d81a-3b4e-4db7-91a1-13a450931085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:30.994079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6d81a-3b4e-4db7-91a1-13a450931085 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:30.994371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6d81a-3b4e-4db7-91a1-13a450931085 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:30.994651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6d81a-3b4e-4db7-91a1-13a450931085 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:30.995493 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6d81a-3b4e-4db7-91a1-13a450931085 None admin] RBAC: Adding query filter params (role.id=f57b3e871ce142a090d6aa7dea9ac7d5, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=a09029144c454b099bea86aaa5229ef0) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:30.995809 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6d81a-3b4e-4db7-91a1-13a450931085 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:30.997286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6d81a-3b4e-4db7-91a1-13a450931085 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:31.003213 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 438/864] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:16:30 2022] GET /identity/v3/role_assignments?user.id=a09029144c454b099bea86aaa5229ef0&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=f57b3e871ce142a090d6aa7dea9ac7d5 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:31.004290 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6490d-d60e-4023-a816-a5e185f1e670 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:31.004468 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1531f0-f4a2-4c89-8294-7f0d9f2a5034 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:31.004791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1531f0-f4a2-4c89-8294-7f0d9f2a5034 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:31.005073 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1531f0-f4a2-4c89-8294-7f0d9f2a5034 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:31.006537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 439/865] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:31.009415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1531f0-f4a2-4c89-8294-7f0d9f2a5034 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:31.043291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc98caf-5b97-483b-89e2-925f61227569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:31.043885 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc98caf-5b97-483b-89e2-925f61227569 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:31.044256 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc98caf-5b97-483b-89e2-925f61227569 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:31.044438 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc98caf-5b97-483b-89e2-925f61227569 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:31.045207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc98caf-5b97-483b-89e2-925f61227569 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:31.046605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc98caf-5b97-483b-89e2-925f61227569 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:31.051009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:31.068716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 427/866] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 18 10:16:30 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:16:31.071969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910044c7-3d47-48d3-b62a-c100e3b6fbe1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:31.072529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910044c7-3d47-48d3-b62a-c100e3b6fbe1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:31.073042 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910044c7-3d47-48d3-b62a-c100e3b6fbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:31.099028 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-910044c7-3d47-48d3-b62a-c100e3b6fbe1 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:31.100743 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-910044c7-3d47-48d3-b62a-c100e3b6fbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:31.140807 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:31.168885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1531f0-f4a2-4c89-8294-7f0d9f2a5034 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:31.197538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 428/867] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:31 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:31.216406 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1471628-d7b1-489e-ad90-7290bd96f248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:31.217257 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1471628-d7b1-489e-ad90-7290bd96f248 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:31.217548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1471628-d7b1-489e-ad90-7290bd96f248 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:31.217829 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1471628-d7b1-489e-ad90-7290bd96f248 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:31.221953 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910044c7-3d47-48d3-b62a-c100e3b6fbe1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:31.231302 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1471628-d7b1-489e-ad90-7290bd96f248 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:31.233398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1471628-d7b1-489e-ad90-7290bd96f248 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:31.252126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a1471628-d7b1-489e-ad90-7290bd96f248 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:31.252126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:31.252126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:31.252126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:31.252126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:31.252126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:31.252126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:31.252126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:31.252126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:31.252126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:31.252126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:31.252126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:31.252126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:31.252126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:31.252126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:31.253186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:31.253186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:31.253186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:31.253186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:31.253186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:31.253186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:31.253186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:31.253186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:31.253186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:31.253186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:31.253186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:31.253186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:31.253186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:31.253186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:31.253186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:31.253186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:31.253186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:31.254296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:31.254296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:31.254296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:31.254296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:31.254296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:31.254296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:31.254296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:31.254296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:31.256426 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 440/868] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:31.261894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1471628-d7b1-489e-ad90-7290bd96f248 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:31.344251 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910044c7-3d47-48d3-b62a-c100e3b6fbe1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:31.374387 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:31.383133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1471628-d7b1-489e-ad90-7290bd96f248 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:31.384992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83275969-428a-4493-b9f6-d96b2cbf73ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:31.386702 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83275969-428a-4493-b9f6-d96b2cbf73ac None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:31.386993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83275969-428a-4493-b9f6-d96b2cbf73ac None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:31.387272 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83275969-428a-4493-b9f6-d96b2cbf73ac None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:31.387936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83275969-428a-4493-b9f6-d96b2cbf73ac None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:31.389329 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83275969-428a-4493-b9f6-d96b2cbf73ac None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:31.405216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-83275969-428a-4493-b9f6-d96b2cbf73ac None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:16:31.405216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:31.405216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:31.405216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:31.405216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:31.405216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:31.405216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:31.405216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:31.405216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:16:31.405216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:31.405216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:31.405216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:31.405216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:31.405216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:31.405216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:31.405216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:31.406442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:31.406442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:31.406442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:31.406442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:31.406442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:31.406442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:31.406442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:31.406442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:31.406442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:31.406442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:31.406442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:31.406442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:31.406442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:31.406442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:31.406442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:31.406442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:31.407517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:31.407517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:31.407517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:31.407517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:31.407517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:31.407517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:31.407517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:31.407517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:31.407517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:31.407517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:31.407517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:31.407517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:31.407517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:31.407517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:16:31.407517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:31.410049 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 429/869] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Jul 18 10:16:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:31.416829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83275969-428a-4493-b9f6-d96b2cbf73ac None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:31.424003 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c54222d-8a9d-4f5b-b055-f1c52715ca97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:31.424701 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54222d-8a9d-4f5b-b055-f1c52715ca97 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:31.424987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54222d-8a9d-4f5b-b055-f1c52715ca97 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:31.425264 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54222d-8a9d-4f5b-b055-f1c52715ca97 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:31.426127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c54222d-8a9d-4f5b-b055-f1c52715ca97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:31.426437 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c54222d-8a9d-4f5b-b055-f1c52715ca97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:31.429091 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c54222d-8a9d-4f5b-b055-f1c52715ca97 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:31.445968 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 441/870] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:31.451631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c54222d-8a9d-4f5b-b055-f1c52715ca97 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:31.462416 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:31.536911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83275969-428a-4493-b9f6-d96b2cbf73ac None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:31.555782 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c54222d-8a9d-4f5b-b055-f1c52715ca97 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:31.576587 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4d25a-bf32-4073-847b-9c11ecc2a566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:31.577269 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4d25a-bf32-4073-847b-9c11ecc2a566 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:31.577556 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4d25a-bf32-4073-847b-9c11ecc2a566 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:31.577835 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4d25a-bf32-4073-847b-9c11ecc2a566 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:31.578552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4d25a-bf32-4073-847b-9c11ecc2a566 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:31.578868 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4d25a-bf32-4073-847b-9c11ecc2a566 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:31.580379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4d25a-bf32-4073-847b-9c11ecc2a566 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:31.590939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 430/871] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 18 10:16:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:31.591802 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4d25a-bf32-4073-847b-9c11ecc2a566 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:31.592866 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38069523-fc0a-4b72-b246-d7d4503d7998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:31.593689 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38069523-fc0a-4b72-b246-d7d4503d7998 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:31.594142 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38069523-fc0a-4b72-b246-d7d4503d7998 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:31.594791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38069523-fc0a-4b72-b246-d7d4503d7998 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:31.596781 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38069523-fc0a-4b72-b246-d7d4503d7998 None admin] RBAC: Adding query filter params (role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=fd6168873f2846c8af73b903bba17eb8) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:31.597252 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38069523-fc0a-4b72-b246-d7d4503d7998 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:31.598862 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38069523-fc0a-4b72-b246-d7d4503d7998 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:31.612709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 442/872] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:16:31 2022] GET /identity/v3/role_assignments?user.id=fd6168873f2846c8af73b903bba17eb8&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:31.672891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:31.712612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4d25a-bf32-4073-847b-9c11ecc2a566 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:31.751377 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc2311-234a-4f62-a220-93a320ff0f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:31.752057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc2311-234a-4f62-a220-93a320ff0f17 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:31.752344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc2311-234a-4f62-a220-93a320ff0f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:31.752659 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc2311-234a-4f62-a220-93a320ff0f17 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:31.753389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc2311-234a-4f62-a220-93a320ff0f17 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:31.753706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc2311-234a-4f62-a220-93a320ff0f17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:31.755197 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc2311-234a-4f62-a220-93a320ff0f17 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:31.760969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 431/873] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 18 10:16:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:33.140722 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a16fe3-45ac-4857-9891-3e2cb2aeffe0 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:33.141244 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a16fe3-45ac-4857-9891-3e2cb2aeffe0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:33.141244 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a16fe3-45ac-4857-9891-3e2cb2aeffe0 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:33.142862 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 443/874] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:33.148244 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7a192-f517-44e6-afc6-e9f8202703f6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:33.148682 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7a192-f517-44e6-afc6-e9f8202703f6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:33.148865 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7a192-f517-44e6-afc6-e9f8202703f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:33.172782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c7d7a192-f517-44e6-afc6-e9f8202703f6 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:33.173972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c7d7a192-f517-44e6-afc6-e9f8202703f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:33.245267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 432/875] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:33.263824 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d73b6-e65a-4c78-89b7-5398806bbc80 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:33.264399 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d73b6-e65a-4c78-89b7-5398806bbc80 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:33.264898 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d73b6-e65a-4c78-89b7-5398806bbc80 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:33.266726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 444/876] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:33.271915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d73b6-e65a-4c78-89b7-5398806bbc80 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:33.272694 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ceaac-b298-43d8-9cba-e12f84ab42cb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:33.273002 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ceaac-b298-43d8-9cba-e12f84ab42cb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:33.273280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ceaac-b298-43d8-9cba-e12f84ab42cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:33.290509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c33ceaac-b298-43d8-9cba-e12f84ab42cb None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:33.291691 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c33ceaac-b298-43d8-9cba-e12f84ab42cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:33.346527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:33.368893 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:33.371647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 433/877] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:16:33.380876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d73b6-e65a-4c78-89b7-5398806bbc80 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:33.392257 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ceaac-b298-43d8-9cba-e12f84ab42cb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:33.417851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054ced2-eadf-41c7-8a2e-6233d452a7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:33.418994 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054ced2-eadf-41c7-8a2e-6233d452a7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:33.419477 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054ced2-eadf-41c7-8a2e-6233d452a7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:33.419964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054ced2-eadf-41c7-8a2e-6233d452a7b3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:33.420862 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054ced2-eadf-41c7-8a2e-6233d452a7b3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:33.422423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054ced2-eadf-41c7-8a2e-6233d452a7b3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:33.434496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7054ced2-eadf-41c7-8a2e-6233d452a7b3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:16:33.434496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:33.434496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:33.434496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:33.434496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:33.434496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:33.434496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:33.434496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:33.434496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:16:33.434496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:33.434496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:33.434496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:33.434496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:33.434496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:33.434496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:33.434496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:33.434496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:33.435501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:33.435501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:33.435501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:33.435501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:33.435501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:33.435501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:33.435501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:33.435501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:33.435501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:33.435501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:33.435501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:33.435501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:33.435501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:33.435501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:33.435501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:33.435501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:33.436325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:33.436325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:33.436325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:33.436325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:33.436325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:33.436325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:33.436325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:33.436325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:33.436325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:33.436325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:33.436325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:33.436325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:33.436325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:16:33.436325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:33.439116 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 445/878] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:16:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:33.445670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054ced2-eadf-41c7-8a2e-6233d452a7b3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:33.498231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ceaac-b298-43d8-9cba-e12f84ab42cb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:33.544289 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed9d6b-918f-4fc8-8d19-b37cd7d18bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:33.545763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed9d6b-918f-4fc8-8d19-b37cd7d18bdd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:33.546657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed9d6b-918f-4fc8-8d19-b37cd7d18bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:33.547136 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed9d6b-918f-4fc8-8d19-b37cd7d18bdd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:33.561564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed9d6b-918f-4fc8-8d19-b37cd7d18bdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:33.561886 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054ced2-eadf-41c7-8a2e-6233d452a7b3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:33.571675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8bed9d6b-918f-4fc8-8d19-b37cd7d18bdd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:16:33.571675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:33.571675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:33.571675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:33.571675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:33.571675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:33.571675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:33.571675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:33.571675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:33.571675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:33.571675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:33.571675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:33.571675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:33.571675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:33.571675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:33.571675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:33.572687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:33.572687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:33.572687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:33.572687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:33.572687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:33.572687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:33.572687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:33.572687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:33.572687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:33.572687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:33.572687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:33.572687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:33.572687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:33.572687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:33.572687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:33.572687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:33.573544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:33.573544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:33.573544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:33.573544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:33.573544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:33.573544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:33.573544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:33.573544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:33.573544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:33.573544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:33.573544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:16:33.573544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:33.576990 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 434/879] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Jul 18 10:16:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:33.579348 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed9d6b-918f-4fc8-8d19-b37cd7d18bdd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:33.612726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478b8b9-0cf3-4ed3-9d95-666b68eb316a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:33.613965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478b8b9-0cf3-4ed3-9d95-666b68eb316a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:33.614474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478b8b9-0cf3-4ed3-9d95-666b68eb316a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:33.614942 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478b8b9-0cf3-4ed3-9d95-666b68eb316a None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:33.618075 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478b8b9-0cf3-4ed3-9d95-666b68eb316a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:33.618645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478b8b9-0cf3-4ed3-9d95-666b68eb316a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:33.620243 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478b8b9-0cf3-4ed3-9d95-666b68eb316a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:33.621646 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:33.623337 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:33.628861 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 446/880] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:16:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:33.635730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478b8b9-0cf3-4ed3-9d95-666b68eb316a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:33.692387 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed9d6b-918f-4fc8-8d19-b37cd7d18bdd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:33.728214 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ec4a3-579a-4913-88a8-f96bace373cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:33.728929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ec4a3-579a-4913-88a8-f96bace373cd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:33.729211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ec4a3-579a-4913-88a8-f96bace373cd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:33.729488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ec4a3-579a-4913-88a8-f96bace373cd None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:33.730486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ec4a3-579a-4913-88a8-f96bace373cd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:33.730797 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ec4a3-579a-4913-88a8-f96bace373cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:33.732229 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ec4a3-579a-4913-88a8-f96bace373cd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:33.743245 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478b8b9-0cf3-4ed3-9d95-666b68eb316a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:33.745680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 435/881] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jul 18 10:16:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:33.747222 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259f2d6-ac6a-4644-b604-b58801fbecc6 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:33.747517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259f2d6-ac6a-4644-b604-b58801fbecc6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:33.747800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259f2d6-ac6a-4644-b604-b58801fbecc6 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:33.749297 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 436/882] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:33.753060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259f2d6-ac6a-4644-b604-b58801fbecc6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:33.782678 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a9086-b7da-4fbb-99e0-b81bcead6160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:33.783456 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a9086-b7da-4fbb-99e0-b81bcead6160 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:33.783738 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a9086-b7da-4fbb-99e0-b81bcead6160 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:33.784023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a9086-b7da-4fbb-99e0-b81bcead6160 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:33.784928 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a9086-b7da-4fbb-99e0-b81bcead6160 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:33.785252 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a9086-b7da-4fbb-99e0-b81bcead6160 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:33.786613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a9086-b7da-4fbb-99e0-b81bcead6160 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:33.796835 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 447/883] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:16:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:33.798663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d7ce6-30bd-4e29-a1f5-2edc4781718e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:33.798962 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d7ce6-30bd-4e29-a1f5-2edc4781718e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:33.799602 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d7ce6-30bd-4e29-a1f5-2edc4781718e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:33.816663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2c5d7ce6-30bd-4e29-a1f5-2edc4781718e None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:33.817855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2c5d7ce6-30bd-4e29-a1f5-2edc4781718e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:33.853604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:33.858015 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259f2d6-ac6a-4644-b604-b58801fbecc6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:33.875246 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:33.893710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 448/884] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:33.893710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d7ce6-30bd-4e29-a1f5-2edc4781718e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:33.903737 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b9e89-dd31-452c-9962-c0d355278eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:33.904644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b9e89-dd31-452c-9962-c0d355278eeb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:33.904934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b9e89-dd31-452c-9962-c0d355278eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:33.905209 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b9e89-dd31-452c-9962-c0d355278eeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:33.916168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b9e89-dd31-452c-9962-c0d355278eeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:33.917664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b9e89-dd31-452c-9962-c0d355278eeb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:33.935065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-167b9e89-dd31-452c-9962-c0d355278eeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:33.935065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:33.935065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:33.935065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:33.935065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:33.935065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:33.935065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:33.935065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:33.935065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:33.935065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:33.935065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:33.935065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:33.935065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:33.935065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:33.935065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:33.935965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:33.935965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:33.935965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:33.935965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:33.935965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:33.935965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:33.935965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:33.935965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:33.935965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:33.935965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:33.935965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:33.935965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:33.935965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:33.935965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:33.935965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:33.935965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:33.935965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:33.937344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:33.937344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:33.937344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:33.937344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:33.937344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 437/885] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:33.937643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9aa4b-77aa-4570-b05e-3a721c7c3d34 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:33.937749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9aa4b-77aa-4570-b05e-3a721c7c3d34 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:33.938029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9aa4b-77aa-4570-b05e-3a721c7c3d34 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:33.939288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 438/886] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:33.939969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9aa4b-77aa-4570-b05e-3a721c7c3d34 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:33.993063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d7ce6-30bd-4e29-a1f5-2edc4781718e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:34.026099 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c004d-9d98-408e-b837-f8f868537b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:34.026781 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c004d-9d98-408e-b837-f8f868537b9a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:34.027067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c004d-9d98-408e-b837-f8f868537b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:34.027344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c004d-9d98-408e-b837-f8f868537b9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:34.028134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c004d-9d98-408e-b837-f8f868537b9a None admin] RBAC: Adding query filter params (interface=public, service_id=940810e717554246a2ddd66100185642, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:34.028446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c004d-9d98-408e-b837-f8f868537b9a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:34.029869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c004d-9d98-408e-b837-f8f868537b9a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:34.034923 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 449/887] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jul 18 10:16:33 2022] GET /identity/v3/endpoints?service_id=940810e717554246a2ddd66100185642&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:34.035692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c004d-9d98-408e-b837-f8f868537b9a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:34.044880 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9aa4b-77aa-4570-b05e-3a721c7c3d34 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:34.080061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62725461-973a-42f1-8e03-933039f1b18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:34.080887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62725461-973a-42f1-8e03-933039f1b18d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:34.081169 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62725461-973a-42f1-8e03-933039f1b18d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:34.081444 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62725461-973a-42f1-8e03-933039f1b18d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:34.082091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62725461-973a-42f1-8e03-933039f1b18d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:34.083441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62725461-973a-42f1-8e03-933039f1b18d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:34.090014 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:34.095760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-62725461-973a-42f1-8e03-933039f1b18d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:16:34.095760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:34.095760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:34.095760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:34.095760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:34.095760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:34.095760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:34.095760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:34.095760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:16:34.095760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:34.095760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:34.095760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:34.095760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:34.095760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:34.095760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:34.095760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:34.095760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:34.096790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:34.096790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:34.096790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:34.096790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:34.096790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:34.096790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:34.096790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:34.096790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:34.096790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:34.096790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:34.096790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:34.096790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:34.096790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:34.096790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:34.096790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:34.096790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:34.097650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:34.097650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:34.097650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:34.097650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:34.097650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:34.097650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:34.097650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:34.097650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:34.097650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:34.097650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:34.097650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:34.097650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:34.097650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:16:34.097650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:34.097650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 439/888] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 18 10:16:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:34.098452 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40eb265-e17f-4586-93a4-2215589c9752 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:34.098528 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40eb265-e17f-4586-93a4-2215589c9752 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:34.099126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40eb265-e17f-4586-93a4-2215589c9752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:34.115362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d40eb265-e17f-4586-93a4-2215589c9752 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:34.116590 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d40eb265-e17f-4586-93a4-2215589c9752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:34.126897 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:34.137393 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c004d-9d98-408e-b837-f8f868537b9a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:34.171450 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e68fe-dbf6-49d1-9165-209e4d0d79ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:34.172114 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e68fe-dbf6-49d1-9165-209e4d0d79ca None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:34.172400 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e68fe-dbf6-49d1-9165-209e4d0d79ca None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:34.172739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e68fe-dbf6-49d1-9165-209e4d0d79ca None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:34.173462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e68fe-dbf6-49d1-9165-209e4d0d79ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:34.174204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e68fe-dbf6-49d1-9165-209e4d0d79ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:34.175576 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e68fe-dbf6-49d1-9165-209e4d0d79ca None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:34.186883 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 440/889] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:34.187382 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 450/890] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:34.187737 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40eb265-e17f-4586-93a4-2215589c9752 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:34.188558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e68fe-dbf6-49d1-9165-209e4d0d79ca None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:34.287778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40eb265-e17f-4586-93a4-2215589c9752 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:34.296427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e68fe-dbf6-49d1-9165-209e4d0d79ca None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:34.312240 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:34.320757 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f587f-2ae6-4410-a238-aa04b0b21f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:34.321416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f587f-2ae6-4410-a238-aa04b0b21f53 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:34.321703 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f587f-2ae6-4410-a238-aa04b0b21f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:34.321984 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f587f-2ae6-4410-a238-aa04b0b21f53 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:34.322668 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f587f-2ae6-4410-a238-aa04b0b21f53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:34.324035 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f587f-2ae6-4410-a238-aa04b0b21f53 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:34.329357 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 441/891] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jul 18 10:16:34 2022] GET /identity/v3/services => generated 1011 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:16:34.330124 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f587f-2ae6-4410-a238-aa04b0b21f53 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:34.337464 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2753708-480e-4ae9-b428-9f61755f0975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:34.338417 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2753708-480e-4ae9-b428-9f61755f0975 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:34.338703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2753708-480e-4ae9-b428-9f61755f0975 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:34.338984 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2753708-480e-4ae9-b428-9f61755f0975 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:34.339724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2753708-480e-4ae9-b428-9f61755f0975 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:34.340036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2753708-480e-4ae9-b428-9f61755f0975 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:34.341484 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2753708-480e-4ae9-b428-9f61755f0975 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:34.347136 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 451/892] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 18 10:16:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:34.348007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2753708-480e-4ae9-b428-9f61755f0975 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:34.355555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:34.458351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f587f-2ae6-4410-a238-aa04b0b21f53 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:34.485230 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2753708-480e-4ae9-b428-9f61755f0975 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:34.501789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503e9a9-2b3e-47b5-82ed-286015a57929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:34.502611 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503e9a9-2b3e-47b5-82ed-286015a57929 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:34.502902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503e9a9-2b3e-47b5-82ed-286015a57929 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:34.503182 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503e9a9-2b3e-47b5-82ed-286015a57929 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:34.512042 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503e9a9-2b3e-47b5-82ed-286015a57929 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:34.513529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503e9a9-2b3e-47b5-82ed-286015a57929 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:34.520975 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f55ed-7a2a-4d2f-9d5b-2e09e0f94a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:34.521639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f55ed-7a2a-4d2f-9d5b-2e09e0f94a8d None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:34.521930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f55ed-7a2a-4d2f-9d5b-2e09e0f94a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:34.522756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f55ed-7a2a-4d2f-9d5b-2e09e0f94a8d None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:34.523496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f55ed-7a2a-4d2f-9d5b-2e09e0f94a8d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:34.524883 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f55ed-7a2a-4d2f-9d5b-2e09e0f94a8d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:34.529565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8503e9a9-2b3e-47b5-82ed-286015a57929 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:16:34.529565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:34.529565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:34.529565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:34.529565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:34.529565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:34.529565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:34.529565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:34.529565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:34.529565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:34.529565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:34.529565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:34.529565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:34.529565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:34.529565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:34.529565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:34.530641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:34.530641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:34.530641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:34.530641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:34.530641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:34.530641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:34.530641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:34.530641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:34.530641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:34.530641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:34.530641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:34.530641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:34.530641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:34.530641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:34.530641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:34.530641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:34.530641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:34.530641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:34.531709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:34.531709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:34.531709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:34.531709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:34.531709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:34.531709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:34.531709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:34.531709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:34.531709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:16:34.531709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:34.531709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 442/893] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 18 10:16:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:34.532300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503e9a9-2b3e-47b5-82ed-286015a57929 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:34.543375 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 452/894] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 18 10:16:34 2022] POST /identity/v3/services => generated 269 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:16:34.544165 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f55ed-7a2a-4d2f-9d5b-2e09e0f94a8d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:34.599778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:34.628764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:34.668103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503e9a9-2b3e-47b5-82ed-286015a57929 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:34.701527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f55ed-7a2a-4d2f-9d5b-2e09e0f94a8d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:34.707478 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c467a-fed9-4941-9adc-26ab5a4e75f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:34.708158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c467a-fed9-4941-9adc-26ab5a4e75f9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:34.708444 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c467a-fed9-4941-9adc-26ab5a4e75f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:34.708762 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c467a-fed9-4941-9adc-26ab5a4e75f9 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:34.709487 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c467a-fed9-4941-9adc-26ab5a4e75f9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:34.709938 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c467a-fed9-4941-9adc-26ab5a4e75f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:34.711323 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c467a-fed9-4941-9adc-26ab5a4e75f9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:34.716445 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 443/895] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 18 10:16:34 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:34.739696 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebe53f-cc99-4a85-8f53-cedb49044104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:34.740391 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebe53f-cc99-4a85-8f53-cedb49044104 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:34.740707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebe53f-cc99-4a85-8f53-cedb49044104 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:34.741542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebe53f-cc99-4a85-8f53-cedb49044104 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:34.742420 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebe53f-cc99-4a85-8f53-cedb49044104 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:34.742729 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebe53f-cc99-4a85-8f53-cedb49044104 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:34.745651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebe53f-cc99-4a85-8f53-cedb49044104 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:34.770606 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 453/896] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Jul 18 10:16:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:34.776288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c467a-fed9-4941-9adc-26ab5a4e75f9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:34.897161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c467a-fed9-4941-9adc-26ab5a4e75f9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:34.934988 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333298f-7d56-40f9-8326-0e58f4e02221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:34.935643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333298f-7d56-40f9-8326-0e58f4e02221 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:34.935928 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333298f-7d56-40f9-8326-0e58f4e02221 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:34.936207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333298f-7d56-40f9-8326-0e58f4e02221 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:34.939242 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:35.010643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333298f-7d56-40f9-8326-0e58f4e02221 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:35.012117 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333298f-7d56-40f9-8326-0e58f4e02221 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:35.119127 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 444/897] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:16:34 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:16:35.124708 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebe53f-cc99-4a85-8f53-cedb49044104 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:35.235237 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:35.242646 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebe53f-cc99-4a85-8f53-cedb49044104 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:35.278616 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0670cf-61fc-4a33-800f-12468425c7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:35.279417 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0670cf-61fc-4a33-800f-12468425c7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:35.279699 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0670cf-61fc-4a33-800f-12468425c7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:35.279973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0670cf-61fc-4a33-800f-12468425c7d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:35.294537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0670cf-61fc-4a33-800f-12468425c7d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:35.296049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0670cf-61fc-4a33-800f-12468425c7d3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:35.314336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-4c0670cf-61fc-4a33-800f-12468425c7d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:35.314336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:35.314336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:35.314336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:35.314336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:35.314336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:35.314336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:35.314336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:35.314336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:35.314336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:35.314336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:35.314336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:35.314336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:35.314336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:35.314336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:35.315325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:35.315325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:35.315325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:35.315325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:35.315325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:35.315325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:35.315325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:35.315325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:35.315325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:35.315325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:35.315325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:35.315325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:35.315325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:35.315325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:35.315325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:35.315325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:35.315325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:35.316233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:35.316233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:35.316233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:35.316233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:35.316233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 454/898] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:35.323150 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333298f-7d56-40f9-8326-0e58f4e02221 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:35.357757 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:35.428092 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333298f-7d56-40f9-8326-0e58f4e02221 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:35.463090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1507c2-b04a-43ac-a83e-b66b2e436357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:35.464082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1507c2-b04a-43ac-a83e-b66b2e436357 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:35.464593 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1507c2-b04a-43ac-a83e-b66b2e436357 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:35.465067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1507c2-b04a-43ac-a83e-b66b2e436357 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:35.465999 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1507c2-b04a-43ac-a83e-b66b2e436357 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:35.466501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1507c2-b04a-43ac-a83e-b66b2e436357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:35.468062 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1507c2-b04a-43ac-a83e-b66b2e436357 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:35.481235 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 445/899] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:35.487107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0670cf-61fc-4a33-800f-12468425c7d3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:35.593605 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0670cf-61fc-4a33-800f-12468425c7d3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:35.628927 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cd556-5a6c-4724-bd72-68d3b47bb23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:35.629781 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cd556-5a6c-4724-bd72-68d3b47bb23c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:35.630069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cd556-5a6c-4724-bd72-68d3b47bb23c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:35.630401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cd556-5a6c-4724-bd72-68d3b47bb23c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:35.637684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cd556-5a6c-4724-bd72-68d3b47bb23c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:35.639118 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cd556-5a6c-4724-bd72-68d3b47bb23c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:35.659041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-4d2cd556-5a6c-4724-bd72-68d3b47bb23c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:35.659041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:35.659041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:35.659041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:35.659041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:35.659041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:35.659041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:35.659041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:35.659041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:35.659041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:35.659041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:35.659041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:35.659041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:35.659041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:35.659041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:35.659952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:35.659952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:35.659952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:35.659952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:35.659952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:35.659952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:35.659952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:35.659952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:35.659952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:35.659952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:35.659952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:35.659952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:35.659952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:35.659952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:35.659952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:35.659952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:35.659952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:35.660902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:35.660902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:35.660902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:35.660902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:35.660902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:35.660902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:35.660902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:35.660902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:35.660902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 455/900] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:35.667569 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1507c2-b04a-43ac-a83e-b66b2e436357 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:35.767899 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:35.775478 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1507c2-b04a-43ac-a83e-b66b2e436357 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:35.811834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7e2c9-3f4b-4c93-9a00-4ceb481f1f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:35.812514 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7e2c9-3f4b-4c93-9a00-4ceb481f1f36 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:35.812813 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7e2c9-3f4b-4c93-9a00-4ceb481f1f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:35.813095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7e2c9-3f4b-4c93-9a00-4ceb481f1f36 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:35.813891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7e2c9-3f4b-4c93-9a00-4ceb481f1f36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:35.814205 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7e2c9-3f4b-4c93-9a00-4ceb481f1f36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:35.815771 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7e2c9-3f4b-4c93-9a00-4ceb481f1f36 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:35.829112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 446/901] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:35.835553 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cd556-5a6c-4724-bd72-68d3b47bb23c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:35.848834 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:35.946617 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cd556-5a6c-4724-bd72-68d3b47bb23c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:35.981326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eaaf20-1bde-4673-a93d-58d1136e3dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:35.981986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eaaf20-1bde-4673-a93d-58d1136e3dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:35.982272 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eaaf20-1bde-4673-a93d-58d1136e3dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:35.982550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eaaf20-1bde-4673-a93d-58d1136e3dd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:35.983380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eaaf20-1bde-4673-a93d-58d1136e3dd9 None admin] RBAC: Adding query filter params (role.id=25bb48ece7784b5d9b0a60db78e81edd, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=a09029144c454b099bea86aaa5229ef0) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:35.983695 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eaaf20-1bde-4673-a93d-58d1136e3dd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:35.985232 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eaaf20-1bde-4673-a93d-58d1136e3dd9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:35.991748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 456/902] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:16:35 2022] GET /identity/v3/role_assignments?user.id=a09029144c454b099bea86aaa5229ef0&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=25bb48ece7784b5d9b0a60db78e81edd => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:36.499191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626e405-1422-4b3b-9962-2198bd41ccf5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:36.499680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626e405-1422-4b3b-9962-2198bd41ccf5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:36.499680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626e405-1422-4b3b-9962-2198bd41ccf5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:36.501377 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 447/903] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:36.507121 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bce57e-cb6e-47d7-b4d5-d7945cee53d9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:36.507531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bce57e-cb6e-47d7-b4d5-d7945cee53d9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:36.507714 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bce57e-cb6e-47d7-b4d5-d7945cee53d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:36.525270 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-30bce57e-cb6e-47d7-b4d5-d7945cee53d9 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:36.526488 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-30bce57e-cb6e-47d7-b4d5-d7945cee53d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:36.592107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:36.597376 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 457/904] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:36.625457 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626e405-1422-4b3b-9962-2198bd41ccf5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:36.713605 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48b4bb-5257-4677-b9b2-e921ab504a92 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:36.713930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48b4bb-5257-4677-b9b2-e921ab504a92 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:36.714170 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48b4bb-5257-4677-b9b2-e921ab504a92 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:36.715825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 458/905] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:36.720696 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a936ba5-48a3-4aa6-bb23-56d486bb172f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:36.720988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a936ba5-48a3-4aa6-bb23-56d486bb172f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:36.721270 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a936ba5-48a3-4aa6-bb23-56d486bb172f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:36.732314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626e405-1422-4b3b-9962-2198bd41ccf5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:36.738896 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1a936ba5-48a3-4aa6-bb23-56d486bb172f None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:36.740137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1a936ba5-48a3-4aa6-bb23-56d486bb172f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:36.776991 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f6941-1109-450f-9807-589092744a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:36.777789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f6941-1109-450f-9807-589092744a5a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:36.778098 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f6941-1109-450f-9807-589092744a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:36.778378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f6941-1109-450f-9807-589092744a5a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:36.779038 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f6941-1109-450f-9807-589092744a5a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:36.780738 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f6941-1109-450f-9807-589092744a5a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:36.793764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c68f6941-1109-450f-9807-589092744a5a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:16:36.793764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:36.793764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:36.793764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:36.793764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:36.793764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:36.793764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:36.793764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:36.793764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:16:36.793764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:36.793764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:36.793764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:36.793764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:36.793764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:36.793764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:36.793764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:36.793764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:36.794798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:36.794798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:36.794798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:36.794798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:36.794798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:36.794798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:36.794798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:36.794798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:36.794798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:36.794798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:36.794798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:36.794798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:36.794798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:36.794798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:36.794798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:36.794798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:36.795624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:36.795624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:36.795624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:36.795624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:36.795624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:36.795624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:36.795624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:36.795624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:36.795624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:36.795624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:36.795624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:36.795624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:36.795624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 10:16:36.795624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:36.795624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 448/906] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:16:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:36.804293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f6941-1109-450f-9807-589092744a5a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:36.839962 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 459/907] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:36.856870 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:36.863725 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15c0df-bdc4-4ec5-a300-3007831285db None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:36.864014 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15c0df-bdc4-4ec5-a300-3007831285db None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:36.864294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15c0df-bdc4-4ec5-a300-3007831285db None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:36.866416 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 460/908] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:16:36.868056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15c0df-bdc4-4ec5-a300-3007831285db None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:36.924442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f6941-1109-450f-9807-589092744a5a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:36.959011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4926f-a98d-41a1-a61a-7d0220b4d17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:36.960221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4926f-a98d-41a1-a61a-7d0220b4d17e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:36.960523 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4926f-a98d-41a1-a61a-7d0220b4d17e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:36.960813 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4926f-a98d-41a1-a61a-7d0220b4d17e None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:36.961519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4926f-a98d-41a1-a61a-7d0220b4d17e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:36.961829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4926f-a98d-41a1-a61a-7d0220b4d17e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:36.963196 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4926f-a98d-41a1-a61a-7d0220b4d17e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:36.968270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 449/909] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:16:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:36.973818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b0eb3-e9c3-4841-a7da-5a10e2458c54 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:36.974111 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b0eb3-e9c3-4841-a7da-5a10e2458c54 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:36.974394 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b0eb3-e9c3-4841-a7da-5a10e2458c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:36.978098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15c0df-bdc4-4ec5-a300-3007831285db None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:36.991930 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-812b0eb3-e9c3-4841-a7da-5a10e2458c54 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:36.993143 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-812b0eb3-e9c3-4841-a7da-5a10e2458c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:37.010836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:37.017602 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb5e4b-7085-4422-bab1-9a400537fa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:37.018535 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb5e4b-7085-4422-bab1-9a400537fa15 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:37.018804 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb5e4b-7085-4422-bab1-9a400537fa15 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:37.019080 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb5e4b-7085-4422-bab1-9a400537fa15 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:37.019738 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb5e4b-7085-4422-bab1-9a400537fa15 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:37.021402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb5e4b-7085-4422-bab1-9a400537fa15 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:37.034704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-1bbb5e4b-7085-4422-bab1-9a400537fa15 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:16:37.034704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:37.034704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:37.034704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:37.034704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:37.034704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:37.034704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:37.034704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:37.034704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:16:37.034704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:37.034704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:37.034704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:37.034704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:37.034704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:37.034704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:37.034704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:37.035726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:37.035726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:37.035726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:37.035726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:37.035726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:37.035726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:37.035726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:37.035726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:37.035726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:37.035726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:37.035726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:37.035726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:37.035726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:37.035726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:37.035726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:37.035726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:37.036621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:37.036621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:37.036621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:37.036621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:37.036621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:37.036621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:37.036621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:37.036621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:37.036621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:37.036621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:37.036621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:37.036621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:37.036621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:37.036621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:16:37.036621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:37.036621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 461/910] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Jul 18 10:16:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:37.037482 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb5e4b-7085-4422-bab1-9a400537fa15 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:37.068369 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 450/911] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:37.069798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b0eb3-e9c3-4841-a7da-5a10e2458c54 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:37.084304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:37.153722 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb5e4b-7085-4422-bab1-9a400537fa15 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:37.188679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c080d4-9646-45c3-9a34-5950809d56b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:37.189344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c080d4-9646-45c3-9a34-5950809d56b3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:37.189628 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c080d4-9646-45c3-9a34-5950809d56b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:37.189903 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c080d4-9646-45c3-9a34-5950809d56b3 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:37.190329 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b0eb3-e9c3-4841-a7da-5a10e2458c54 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:37.190612 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c080d4-9646-45c3-9a34-5950809d56b3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:37.190932 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c080d4-9646-45c3-9a34-5950809d56b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:37.192281 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c080d4-9646-45c3-9a34-5950809d56b3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:37.197315 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 462/912] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jul 18 10:16:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:37.202387 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c080d4-9646-45c3-9a34-5950809d56b3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:37.227323 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918224c6-9584-46b7-a987-7ccdaf14dad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:37.227981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918224c6-9584-46b7-a987-7ccdaf14dad7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:37.228293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918224c6-9584-46b7-a987-7ccdaf14dad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:37.228565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918224c6-9584-46b7-a987-7ccdaf14dad7 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:37.229279 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918224c6-9584-46b7-a987-7ccdaf14dad7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:37.229586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918224c6-9584-46b7-a987-7ccdaf14dad7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:37.231430 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918224c6-9584-46b7-a987-7ccdaf14dad7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:37.240873 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 451/913] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 18 10:16:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:37.240873 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918224c6-9584-46b7-a987-7ccdaf14dad7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:37.267743 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:37.317024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c080d4-9646-45c3-9a34-5950809d56b3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:37.321292 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:37.346553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918224c6-9584-46b7-a987-7ccdaf14dad7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:37.353646 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b854ca-8a37-472d-a14c-cff0c3025e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:37.354821 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b854ca-8a37-472d-a14c-cff0c3025e6a None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:37.355119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b854ca-8a37-472d-a14c-cff0c3025e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:37.355660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b854ca-8a37-472d-a14c-cff0c3025e6a None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:37.356876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b854ca-8a37-472d-a14c-cff0c3025e6a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:37.358954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b854ca-8a37-472d-a14c-cff0c3025e6a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:37.375403 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 463/914] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 18 10:16:37 2022] POST /identity/v3/services => generated 241 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:16:37.376249 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b854ca-8a37-472d-a14c-cff0c3025e6a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:37.400146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d961c6d-9c3b-4538-b0b0-98cf308edeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:37.401514 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d961c6d-9c3b-4538-b0b0-98cf308edeab None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:37.402025 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d961c6d-9c3b-4538-b0b0-98cf308edeab None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:37.402751 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d961c6d-9c3b-4538-b0b0-98cf308edeab None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:37.404234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d961c6d-9c3b-4538-b0b0-98cf308edeab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:37.406000 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d961c6d-9c3b-4538-b0b0-98cf308edeab None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:37.440540 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 452/915] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jul 18 10:16:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:16:37.511836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b854ca-8a37-472d-a14c-cff0c3025e6a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:37.533237 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:37.563238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b436abe-fea1-4e5c-a524-0d9300de956b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:37.563974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b436abe-fea1-4e5c-a524-0d9300de956b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:37.564264 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b436abe-fea1-4e5c-a524-0d9300de956b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:37.564631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b436abe-fea1-4e5c-a524-0d9300de956b None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:37.565287 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b436abe-fea1-4e5c-a524-0d9300de956b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:37.565591 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b436abe-fea1-4e5c-a524-0d9300de956b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:37.567004 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b436abe-fea1-4e5c-a524-0d9300de956b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:37.573868 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 464/916] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 18 10:16:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:37.580484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d961c6d-9c3b-4538-b0b0-98cf308edeab None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:37.695080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d961c6d-9c3b-4538-b0b0-98cf308edeab None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:37.722717 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:37.732394 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84790922-92ac-4691-a0e6-43c5f46311d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:37.733330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84790922-92ac-4691-a0e6-43c5f46311d4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:37.733626 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84790922-92ac-4691-a0e6-43c5f46311d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:37.733974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84790922-92ac-4691-a0e6-43c5f46311d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:37.735028 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84790922-92ac-4691-a0e6-43c5f46311d4 None admin] RBAC: Adding query filter params (interface=public, service_id=b8c41a92e7634c4691910a8d7800f1bd, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:37.735617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84790922-92ac-4691-a0e6-43c5f46311d4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:37.737060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84790922-92ac-4691-a0e6-43c5f46311d4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:37.742590 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 453/917] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jul 18 10:16:37 2022] GET /identity/v3/endpoints?service_id=b8c41a92e7634c4691910a8d7800f1bd&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:37.747908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b436abe-fea1-4e5c-a524-0d9300de956b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:37.852271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b436abe-fea1-4e5c-a524-0d9300de956b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:37.890930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ae0c9-e137-4928-9b36-3af9352593dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:37.892020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ae0c9-e137-4928-9b36-3af9352593dd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:37.892633 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ae0c9-e137-4928-9b36-3af9352593dd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:37.893137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ae0c9-e137-4928-9b36-3af9352593dd None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:37.894022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ae0c9-e137-4928-9b36-3af9352593dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:37.895613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ae0c9-e137-4928-9b36-3af9352593dd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:37.903060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 465/918] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jul 18 10:16:37 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:16:38.030960 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff48e0-fac1-4b0a-a5d9-29d8dfa2b1b5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:38.031260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff48e0-fac1-4b0a-a5d9-29d8dfa2b1b5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:38.031526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff48e0-fac1-4b0a-a5d9-29d8dfa2b1b5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:38.033234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 454/919] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:38.038890 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da907175-dd6b-4396-8489-16ed02c2df33 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:38.039181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da907175-dd6b-4396-8489-16ed02c2df33 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:38.039462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da907175-dd6b-4396-8489-16ed02c2df33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:38.057059 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-da907175-dd6b-4396-8489-16ed02c2df33 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:38.058264 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-da907175-dd6b-4396-8489-16ed02c2df33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:38.098296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:38.137234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 466/920] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:38.164163 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff48e0-fac1-4b0a-a5d9-29d8dfa2b1b5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:38.281825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff48e0-fac1-4b0a-a5d9-29d8dfa2b1b5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:38.322250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d644c-878a-41e6-9a99-7f7fcfad6b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:38.323071 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d644c-878a-41e6-9a99-7f7fcfad6b15 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:38.323353 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d644c-878a-41e6-9a99-7f7fcfad6b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:38.323630 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d644c-878a-41e6-9a99-7f7fcfad6b15 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:38.337998 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d644c-878a-41e6-9a99-7f7fcfad6b15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:38.346278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-137d644c-878a-41e6-9a99-7f7fcfad6b15 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:16:38.346278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:38.346278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:38.346278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:38.346278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:38.346278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:38.346278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:38.346278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:38.346278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:38.346278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:38.346278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:38.346278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:38.346278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:38.346278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:38.346278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:38.346278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:38.347326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:38.347326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:38.347326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:38.347326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:38.347326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:38.347326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:38.347326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:38.347326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:38.347326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:38.347326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:38.347326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:38.347326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:38.347326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:38.347326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:38.347326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:38.347326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:38.348249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:38.348249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:38.348249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:38.348249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:38.348249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:38.348249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:38.348249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:38.348249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:38.348249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:38.348249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:38.348249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:16:38.348249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:38.348249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 455/921] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Jul 18 10:16:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:38.354312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da907175-dd6b-4396-8489-16ed02c2df33 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:38.471751 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da907175-dd6b-4396-8489-16ed02c2df33 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:38.509610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe4582-5d99-4c61-843b-d8ecf01928db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:38.510680 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe4582-5d99-4c61-843b-d8ecf01928db None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:38.511177 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe4582-5d99-4c61-843b-d8ecf01928db None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:38.511659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe4582-5d99-4c61-843b-d8ecf01928db None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:38.512678 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe4582-5d99-4c61-843b-d8ecf01928db None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:38.513725 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe4582-5d99-4c61-843b-d8ecf01928db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:38.515306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe4582-5d99-4c61-843b-d8ecf01928db None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:38.524857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 467/922] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jul 18 10:16:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:38.534106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d644c-878a-41e6-9a99-7f7fcfad6b15 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:38.599168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:38.653800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d644c-878a-41e6-9a99-7f7fcfad6b15 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:38.700226 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62fd51-98ee-464b-83cc-0d7917b63344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:38.701095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62fd51-98ee-464b-83cc-0d7917b63344 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:38.701491 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62fd51-98ee-464b-83cc-0d7917b63344 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:38.701772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62fd51-98ee-464b-83cc-0d7917b63344 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:38.714907 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62fd51-98ee-464b-83cc-0d7917b63344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:38.716427 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62fd51-98ee-464b-83cc-0d7917b63344 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:38.742770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0b62fd51-98ee-464b-83cc-0d7917b63344 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:38.742770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:38.742770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:38.742770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:38.742770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:38.742770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:38.742770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:38.742770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:38.742770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:38.742770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:38.742770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:38.742770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:38.742770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:38.742770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:38.742770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:38.743723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:38.743723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:38.743723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:38.743723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:38.743723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:38.743723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:38.743723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:38.743723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:38.743723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:38.743723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:38.743723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:38.743723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:38.743723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:38.743723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:38.743723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:38.743723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:38.743723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:38.745257 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:38.745257 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:38.745257 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:38.745257 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:38.752769 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 456/923] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:38.754673 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe4582-5d99-4c61-843b-d8ecf01928db None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:38.797763 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:38.886624 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe4582-5d99-4c61-843b-d8ecf01928db None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:38.924541 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931aba1-5f78-4df0-a5d6-719e56436c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:38.925576 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931aba1-5f78-4df0-a5d6-719e56436c6e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:38.926177 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931aba1-5f78-4df0-a5d6-719e56436c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:38.926828 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931aba1-5f78-4df0-a5d6-719e56436c6e None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:38.928883 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931aba1-5f78-4df0-a5d6-719e56436c6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:38.929655 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931aba1-5f78-4df0-a5d6-719e56436c6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:38.931405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931aba1-5f78-4df0-a5d6-719e56436c6e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:38.943982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 468/924] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:38.952537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62fd51-98ee-464b-83cc-0d7917b63344 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:39.056571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62fd51-98ee-464b-83cc-0d7917b63344 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:39.058882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:39.091164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3337d-f109-44f8-85ff-c7f6b9195308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:39.091972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3337d-f109-44f8-85ff-c7f6b9195308 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:39.092254 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3337d-f109-44f8-85ff-c7f6b9195308 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:39.092554 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3337d-f109-44f8-85ff-c7f6b9195308 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:39.100952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af3337d-f109-44f8-85ff-c7f6b9195308 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:39.102464 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af3337d-f109-44f8-85ff-c7f6b9195308 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:39.117496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8af3337d-f109-44f8-85ff-c7f6b9195308 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:16:39.117496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:39.117496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:39.117496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:39.117496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:39.117496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:39.117496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:39.117496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:39.117496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:39.117496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:39.117496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:39.117496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:39.117496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:39.117496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:39.117496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:39.117496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:39.118479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:39.118479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:39.118479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:39.118479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:39.118479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:39.118479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:39.118479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:39.118479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:39.118479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:39.118479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:39.118479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:39.118479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:39.118479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:39.118479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:39.118479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:39.118479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:39.118479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:39.118479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:39.119453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:39.119453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:39.119453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:39.119453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:39.119453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:39.119453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:39.119453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:39.119453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:39.119453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:16:39.119453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:39.119453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 457/925] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 18 10:16:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:39.124752 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931aba1-5f78-4df0-a5d6-719e56436c6e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:39.224474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:39.233053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931aba1-5f78-4df0-a5d6-719e56436c6e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:39.269097 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4f52b-c493-439f-b24a-a9d05ae88872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:39.270174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4f52b-c493-439f-b24a-a9d05ae88872 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:39.270707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4f52b-c493-439f-b24a-a9d05ae88872 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:39.271195 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4f52b-c493-439f-b24a-a9d05ae88872 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:39.272800 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c4f52b-c493-439f-b24a-a9d05ae88872 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:39.273332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c4f52b-c493-439f-b24a-a9d05ae88872 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:39.274955 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c4f52b-c493-439f-b24a-a9d05ae88872 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:39.294498 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 469/926] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Jul 18 10:16:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:39.304547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3337d-f109-44f8-85ff-c7f6b9195308 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:39.403810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3337d-f109-44f8-85ff-c7f6b9195308 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:39.437396 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf80307-58b1-4241-9a85-30fedbd98537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:39.438195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf80307-58b1-4241-9a85-30fedbd98537 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:39.438474 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf80307-58b1-4241-9a85-30fedbd98537 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:39.438750 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf80307-58b1-4241-9a85-30fedbd98537 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:39.450251 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf80307-58b1-4241-9a85-30fedbd98537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:39.451707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf80307-58b1-4241-9a85-30fedbd98537 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:39.468793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-bbf80307-58b1-4241-9a85-30fedbd98537 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:39.468793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:39.468793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:39.468793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:39.468793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:39.468793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:39.468793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:39.468793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:39.468793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:39.468793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:39.468793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:39.468793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:39.468793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:39.468793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:39.468793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:39.469712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:39.469712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:39.469712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:39.469712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:39.469712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:39.469712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:39.469712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:39.469712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:39.469712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:39.469712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:39.469712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:39.469712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:39.469712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:39.469712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:39.469712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:39.469712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:39.469712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:39.470614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:39.470614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:39.470614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:39.470614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:39.470614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 458/927] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:39.476464 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4f52b-c493-439f-b24a-a9d05ae88872 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:39.582746 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4f52b-c493-439f-b24a-a9d05ae88872 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:39.620053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f938e-8fa0-4dbd-a0aa-25b394ac9bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:39.621156 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f938e-8fa0-4dbd-a0aa-25b394ac9bcf None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:39.621715 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f938e-8fa0-4dbd-a0aa-25b394ac9bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:39.622224 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f938e-8fa0-4dbd-a0aa-25b394ac9bcf None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:39.623152 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f938e-8fa0-4dbd-a0aa-25b394ac9bcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:39.623670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f938e-8fa0-4dbd-a0aa-25b394ac9bcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:39.625264 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f938e-8fa0-4dbd-a0aa-25b394ac9bcf None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:39.637638 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 470/928] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:39.648566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf80307-58b1-4241-9a85-30fedbd98537 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:39.668991 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:39.673377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006da71-2e54-4a73-8fe5-69dda3044d03 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:39.673673 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006da71-2e54-4a73-8fe5-69dda3044d03 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:39.673959 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006da71-2e54-4a73-8fe5-69dda3044d03 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:39.676985 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 471/929] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:16:39.682111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d47c07-4a06-4817-813c-1c9ce0a6b490 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:39.682401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d47c07-4a06-4817-813c-1c9ce0a6b490 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:39.682682 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d47c07-4a06-4817-813c-1c9ce0a6b490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:39.699356 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d6d47c07-4a06-4817-813c-1c9ce0a6b490 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:39.700566 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d6d47c07-4a06-4817-813c-1c9ce0a6b490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:39.708856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:39.753484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf80307-58b1-4241-9a85-30fedbd98537 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:39.775313 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 472/930] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:16:39.778142 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189ea30-5783-453f-91dd-d99487a0201a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:39.778673 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189ea30-5783-453f-91dd-d99487a0201a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:39.779406 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189ea30-5783-453f-91dd-d99487a0201a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:39.785684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 473/931] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:39 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:39.787057 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746de387-0dc3-49cf-9680-64a78154a460 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:39.787551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746de387-0dc3-49cf-9680-64a78154a460 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:39.788085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746de387-0dc3-49cf-9680-64a78154a460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:39.795298 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f447d-c28b-4145-adc1-110733ab9c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:39.796151 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f447d-c28b-4145-adc1-110733ab9c17 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:39.796438 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f447d-c28b-4145-adc1-110733ab9c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:39.796750 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f447d-c28b-4145-adc1-110733ab9c17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:39.804786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f447d-c28b-4145-adc1-110733ab9c17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:39.806267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f447d-c28b-4145-adc1-110733ab9c17 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:39.806865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-746de387-0dc3-49cf-9680-64a78154a460 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:39.808256 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-746de387-0dc3-49cf-9680-64a78154a460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:39.821355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-442f447d-c28b-4145-adc1-110733ab9c17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:39.821355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:39.821355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:39.821355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:39.821355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:39.821355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:39.821355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:39.821355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:39.821355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:39.821355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:39.821355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:39.821355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:39.821355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:39.821355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:39.821355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:39.822287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:39.822287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:39.822287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:39.822287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:39.822287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:39.822287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:39.822287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:39.822287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:39.822287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:39.822287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:39.822287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:39.822287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:39.822287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:39.822287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:39.822287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:39.822287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:39.822287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:39.823170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:39.823170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:39.823170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:39.823170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:39.823170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:39.823170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:39.823170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:39.823170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:39.823170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 459/932] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:39.823676 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f447d-c28b-4145-adc1-110733ab9c17 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:39.885755 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 474/933] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:39.886607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746de387-0dc3-49cf-9680-64a78154a460 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:39.921361 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:39.948689 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f447d-c28b-4145-adc1-110733ab9c17 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:39.975488 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:39.986611 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0043a3d-bf67-4545-9bcf-4a211eb057dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:39.987481 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0043a3d-bf67-4545-9bcf-4a211eb057dc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:39.987776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0043a3d-bf67-4545-9bcf-4a211eb057dc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:39.988153 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0043a3d-bf67-4545-9bcf-4a211eb057dc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:39.988925 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0043a3d-bf67-4545-9bcf-4a211eb057dc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:39.990185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0043a3d-bf67-4545-9bcf-4a211eb057dc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:40.002798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a0043a3d-bf67-4545-9bcf-4a211eb057dc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:16:40.002798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:40.002798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:40.002798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:40.002798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:40.002798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:40.002798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:40.002798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:40.002798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:16:40.002798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:40.002798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:40.002798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:40.002798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:40.002798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:40.002798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:40.002798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:40.002798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:40.003827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:40.003827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:40.003827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:40.003827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:40.003827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:40.003827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:40.003827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:40.003827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:40.003827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:40.003827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:40.003827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:40.003827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:40.003827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:40.003827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:40.003827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:40.003827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:40.005450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:40.005450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:40.005450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:40.005450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:40.005450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:40.005450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:40.005450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:40.005450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:40.005450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:40.005450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:40.005450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:40.005450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:40.005450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:16:40.005450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:40.005450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 460/934] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 18 10:16:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:40.006265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0043a3d-bf67-4545-9bcf-4a211eb057dc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:40.011975 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746de387-0dc3-49cf-9680-64a78154a460 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:40.050062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5fd1f-a6af-4c18-98d3-c3e8965acc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:40.050725 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5fd1f-a6af-4c18-98d3-c3e8965acc25 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:40.051011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5fd1f-a6af-4c18-98d3-c3e8965acc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:40.051287 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5fd1f-a6af-4c18-98d3-c3e8965acc25 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:40.052146 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d5fd1f-a6af-4c18-98d3-c3e8965acc25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:40.052457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d5fd1f-a6af-4c18-98d3-c3e8965acc25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:40.054452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d5fd1f-a6af-4c18-98d3-c3e8965acc25 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:40.066406 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 475/935] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:40.069965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5fd1f-a6af-4c18-98d3-c3e8965acc25 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:40.125607 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0043a3d-bf67-4545-9bcf-4a211eb057dc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:40.166010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdcf7a-059d-44a3-96ee-0eecb0cdf0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:40.166674 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdcf7a-059d-44a3-96ee-0eecb0cdf0b3 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:40.166957 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdcf7a-059d-44a3-96ee-0eecb0cdf0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:40.167237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdcf7a-059d-44a3-96ee-0eecb0cdf0b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:40.167938 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdcf7a-059d-44a3-96ee-0eecb0cdf0b3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:40.169392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdcf7a-059d-44a3-96ee-0eecb0cdf0b3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:40.170774 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: WARNING keystone.server.flask.common [None req-30cdcf7a-059d-44a3-96ee-0eecb0cdf0b3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 18 10:16:40.175502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5fd1f-a6af-4c18-98d3-c3e8965acc25 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:40.181642 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:40.189262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.sql.core [None req-30cdcf7a-059d-44a3-96ee-0eecb0cdf0b3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 10:16:40.189262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 10:16:40.189262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [parameters: {'id': '5183238594db487facff841f9c0c7082', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 10:16:40.189262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80142) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 10:16:40.196712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-30cdcf7a-059d-44a3-96ee-0eecb0cdf0b3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5183238594db487facff841f9c0c7082.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5183238594db487facff841f9c0c7082. Jul 18 10:16:40.196712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:40.196712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 10:16:40.196712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 10:16:40.196712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 10:16:40.196712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 10:16:40.196712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 10:16:40.196712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 10:16:40.196712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 10:16:40.196712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application conn.query(q) Jul 18 10:16:40.196712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 10:16:40.196712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 10:16:40.196712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 10:16:40.197719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result.read() Jul 18 10:16:40.197719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 10:16:40.197719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 10:16:40.197719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 10:16:40.197719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 10:16:40.197719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 10:16:40.197719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 10:16:40.197719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 10:16:40.197719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 10:16:40.197719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 10:16:40.197719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:40.197719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 10:16:40.197719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:40.197719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:40.197719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 10:16:40.197719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 10:16:40.197719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 18 10:16:40.197719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 18 10:16:40.198749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 10:16:40.198749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application next(self.gen) Jul 18 10:16:40.198749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 10:16:40.198749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application yield resource Jul 18 10:16:40.198749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 10:16:40.198749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application next(self.gen) Jul 18 10:16:40.198749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 10:16:40.198749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 10:16:40.198749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 10:16:40.198749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 10:16:40.198749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 10:16:40.198749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise self.value Jul 18 10:16:40.198749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 10:16:40.198749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 10:16:40.198749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 10:16:40.198749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application session.commit() Jul 18 10:16:40.198749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 10:16:40.199664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 10:16:40.199664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 10:16:40.199664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 10:16:40.199664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 10:16:40.199664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.session.flush() Jul 18 10:16:40.199664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 10:16:40.199664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 10:16:40.199664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 10:16:40.199664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 10:16:40.199664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 10:16:40.199664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application compat.raise_( Jul 18 10:16:40.199664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 10:16:40.199664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception Jul 18 10:16:40.199664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 10:16:40.199664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 10:16:40.199664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 10:16:40.199664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 10:16:40.200733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 10:16:40.200733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 10:16:40.200733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 10:16:40.200733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 10:16:40.200733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 10:16:40.200733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 10:16:40.200733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 10:16:40.200733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 10:16:40.200733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 10:16:40.200733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 10:16:40.200733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 10:16:40.200733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 10:16:40.200733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 10:16:40.200733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 10:16:40.200733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 10:16:40.201778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 10:16:40.201778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 10:16:40.201778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception Jul 18 10:16:40.201778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 10:16:40.201778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 10:16:40.201778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 10:16:40.201778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 10:16:40.201778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 10:16:40.201778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 10:16:40.201778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 10:16:40.201778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application conn.query(q) Jul 18 10:16:40.201778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 10:16:40.201778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 10:16:40.201778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 10:16:40.201778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result.read() Jul 18 10:16:40.201778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 10:16:40.201778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 10:16:40.203184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 10:16:40.203184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 10:16:40.203184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 10:16:40.203184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 10:16:40.203184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 10:16:40.203184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 10:16:40.203184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 10:16:40.203184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 10:16:40.203184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [parameters: {'id': '5183238594db487facff841f9c0c7082', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 10:16:40.203184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 10:16:40.203184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:40.203184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:40.203184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:40.203184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:40.203184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 18 10:16:40.204560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 18 10:16:40.204560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 10:16:40.204560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 10:16:40.204560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 18 10:16:40.204560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:40.204560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:40.204560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:40.204560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:40.204560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:40.204560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:40.204560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:40.204560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:40.204560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:40.204560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:40.204560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:40.204560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:40.204560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:40.206801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:40.206801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 18 10:16:40.206801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 18 10:16:40.206801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:40.206801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:40.206801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 18 10:16:40.206801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 18 10:16:40.206801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5183238594db487facff841f9c0c7082. Jul 18 10:16:40.206801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:40.206801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 461/936] 10.222.0.33 () {60 vars in 1140 bytes} [Mon Jul 18 10:16:40 2022] POST /identity/v3/projects => generated 262 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:40.206801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdcf7a-059d-44a3-96ee-0eecb0cdf0b3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:40.209822 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:40.215881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ab3a5-b577-46d7-96a6-437a50bc84cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:40.216577 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ab3a5-b577-46d7-96a6-437a50bc84cf None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:40.216868 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ab3a5-b577-46d7-96a6-437a50bc84cf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:40.217144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ab3a5-b577-46d7-96a6-437a50bc84cf None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:40.217861 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ab3a5-b577-46d7-96a6-437a50bc84cf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:40.218169 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ab3a5-b577-46d7-96a6-437a50bc84cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:40.219537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ab3a5-b577-46d7-96a6-437a50bc84cf None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:40.224555 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 476/937] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 18 10:16:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:40.226342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020967af-09e0-4141-8a53-8717fc0dc50c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:40.226633 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020967af-09e0-4141-8a53-8717fc0dc50c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:40.226908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020967af-09e0-4141-8a53-8717fc0dc50c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:40.228294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 477/938] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:40.228924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020967af-09e0-4141-8a53-8717fc0dc50c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:40.305670 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdcf7a-059d-44a3-96ee-0eecb0cdf0b3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:40.329136 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020967af-09e0-4141-8a53-8717fc0dc50c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:40.339347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc65c4-c4c0-4e1e-bc5e-057bf5e9ee5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:40.340221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc65c4-c4c0-4e1e-bc5e-057bf5e9ee5d None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:40.340520 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc65c4-c4c0-4e1e-bc5e-057bf5e9ee5d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:40.340803 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc65c4-c4c0-4e1e-bc5e-057bf5e9ee5d None admin] PATH_INFO: `/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/a09029144c454b099bea86aaa5229ef0/roles/25bb48ece7784b5d9b0a60db78e81edd` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:40.343067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc65c4-c4c0-4e1e-bc5e-057bf5e9ee5d None admin] RBAC: Authorizing `identity:create_grant(project_id=e71a3acd0adf46a2a35b965a27c4998f, user_id=a09029144c454b099bea86aaa5229ef0, role_id=25bb48ece7784b5d9b0a60db78e81edd)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:40.344703 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc65c4-c4c0-4e1e-bc5e-057bf5e9ee5d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:40.372546 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 462/939] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:16:40 2022] PUT /identity/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/a09029144c454b099bea86aaa5229ef0/roles/25bb48ece7784b5d9b0a60db78e81edd => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:16:40.372546 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b8e7e-32bb-4082-8e84-91cdc6f69234 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:40.372546 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b8e7e-32bb-4082-8e84-91cdc6f69234 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:40.372546 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b8e7e-32bb-4082-8e84-91cdc6f69234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:40.373019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734312b0-bbd0-433e-b283-49dcfbd17b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:40.373019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734312b0-bbd0-433e-b283-49dcfbd17b6d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:40.373019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734312b0-bbd0-433e-b283-49dcfbd17b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:40.373019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734312b0-bbd0-433e-b283-49dcfbd17b6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:40.374058 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734312b0-bbd0-433e-b283-49dcfbd17b6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:40.375562 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734312b0-bbd0-433e-b283-49dcfbd17b6d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:40.379347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-950b8e7e-32bb-4082-8e84-91cdc6f69234 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:40.380995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-950b8e7e-32bb-4082-8e84-91cdc6f69234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:40.391017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-734312b0-bbd0-433e-b283-49dcfbd17b6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:40.391017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:40.391017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:40.391017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:40.391017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:40.391017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:40.391017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:40.391017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:40.391017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:40.391017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:40.391017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:40.391017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:40.391017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:40.391017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:40.391017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:40.391982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:40.391982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:40.391982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:40.391982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:40.391982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:40.391982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:40.391982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:40.391982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:40.391982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:40.391982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:40.391982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:40.391982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:40.391982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:40.391982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:40.391982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:40.391982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:40.391982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:40.392908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:40.392908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:40.392908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:40.392908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:40.392908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:40.392908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:40.392908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:40.392908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:40.392908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 478/940] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:40.393402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734312b0-bbd0-433e-b283-49dcfbd17b6d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:40.446205 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:40.459479 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:40.466059 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 463/941] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:40.477522 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b8e7e-32bb-4082-8e84-91cdc6f69234 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:40.511267 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734312b0-bbd0-433e-b283-49dcfbd17b6d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:40.552430 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e636d-3991-42d4-9ff0-21f287cb204f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:40.553315 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e636d-3991-42d4-9ff0-21f287cb204f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:40.553602 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e636d-3991-42d4-9ff0-21f287cb204f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:40.553881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e636d-3991-42d4-9ff0-21f287cb204f None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:40.554652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e636d-3991-42d4-9ff0-21f287cb204f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:40.555073 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e636d-3991-42d4-9ff0-21f287cb204f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:40.556597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e636d-3991-42d4-9ff0-21f287cb204f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:40.565019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 479/942] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 18 10:16:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:40.565019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e636d-3991-42d4-9ff0-21f287cb204f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:40.592278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b8e7e-32bb-4082-8e84-91cdc6f69234 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:40.628712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b353ed-097d-48ee-9a21-2d92200ceba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:40.629479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b353ed-097d-48ee-9a21-2d92200ceba2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:40.629662 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b353ed-097d-48ee-9a21-2d92200ceba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:40.629937 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b353ed-097d-48ee-9a21-2d92200ceba2 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:40.630694 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b353ed-097d-48ee-9a21-2d92200ceba2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:40.631003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b353ed-097d-48ee-9a21-2d92200ceba2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:40.632495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b353ed-097d-48ee-9a21-2d92200ceba2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:40.652897 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 464/943] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 18 10:16:40 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:40.653733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b353ed-097d-48ee-9a21-2d92200ceba2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:40.683629 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e636d-3991-42d4-9ff0-21f287cb204f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:40.715080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:40.720248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:40.741884 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4b0c5-2b1f-4c5a-ac52-bf3641c80bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:40.743281 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4b0c5-2b1f-4c5a-ac52-bf3641c80bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:40.744403 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4b0c5-2b1f-4c5a-ac52-bf3641c80bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:40.744909 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4b0c5-2b1f-4c5a-ac52-bf3641c80bb3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:40.746053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4b0c5-2b1f-4c5a-ac52-bf3641c80bb3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:40.749021 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4b0c5-2b1f-4c5a-ac52-bf3641c80bb3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:40.766499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-00d4b0c5-2b1f-4c5a-ac52-bf3641c80bb3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:16:40.766499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:40.766499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:40.766499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:40.766499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:40.766499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:40.766499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:40.766499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:40.766499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:16:40.766499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:40.766499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:40.766499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:40.766499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:40.766499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:40.766499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:40.766499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:40.767542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:40.767542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:40.767542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:40.767542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:40.767542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:40.767542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:40.767542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:40.767542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:40.767542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:40.767542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:40.767542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:40.767542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:40.767542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:40.767542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:40.767542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:40.767542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:40.768380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:40.768380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:40.768380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:40.768380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:40.768380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:40.768380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:40.768380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:40.768380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:40.768380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:40.768380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:40.768380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:40.768380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:40.768380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:40.768380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 10:16:40.768380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:40.772652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 480/944] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Jul 18 10:16:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:40.776606 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4b0c5-2b1f-4c5a-ac52-bf3641c80bb3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:40.791329 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b353ed-097d-48ee-9a21-2d92200ceba2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:40.851343 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a058f72-89c6-454b-9ded-615ab14ba0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:40.851981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a058f72-89c6-454b-9ded-615ab14ba0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:40.852261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a058f72-89c6-454b-9ded-615ab14ba0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:40.852736 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a058f72-89c6-454b-9ded-615ab14ba0b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:40.853356 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a058f72-89c6-454b-9ded-615ab14ba0b4 None admin] RBAC: Adding query filter params (interface=public, service_id=baa482f55c1e4bf491ab6f1092ff9f54, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:40.853667 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a058f72-89c6-454b-9ded-615ab14ba0b4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:40.855039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a058f72-89c6-454b-9ded-615ab14ba0b4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:40.861074 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 465/945] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jul 18 10:16:40 2022] GET /identity/v3/endpoints?service_id=baa482f55c1e4bf491ab6f1092ff9f54&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:40.866450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a058f72-89c6-454b-9ded-615ab14ba0b4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:40.910459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4b0c5-2b1f-4c5a-ac52-bf3641c80bb3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:40.951007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be945c96-c2d4-4712-bf52-5ab7ab93117c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:40.951673 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be945c96-c2d4-4712-bf52-5ab7ab93117c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:40.951954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be945c96-c2d4-4712-bf52-5ab7ab93117c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:40.952227 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be945c96-c2d4-4712-bf52-5ab7ab93117c None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:40.952969 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be945c96-c2d4-4712-bf52-5ab7ab93117c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:40.953282 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be945c96-c2d4-4712-bf52-5ab7ab93117c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:40.954644 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be945c96-c2d4-4712-bf52-5ab7ab93117c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:40.960695 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 481/946] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 18 10:16:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:40.969570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be945c96-c2d4-4712-bf52-5ab7ab93117c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:40.986161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a058f72-89c6-454b-9ded-615ab14ba0b4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:41.000001 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:41.031039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58dd60d-7ef7-4a7b-8cda-dfe1521f765d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:41.031877 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58dd60d-7ef7-4a7b-8cda-dfe1521f765d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:41.032257 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58dd60d-7ef7-4a7b-8cda-dfe1521f765d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:41.032710 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58dd60d-7ef7-4a7b-8cda-dfe1521f765d None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:41.033722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58dd60d-7ef7-4a7b-8cda-dfe1521f765d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:41.035679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58dd60d-7ef7-4a7b-8cda-dfe1521f765d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:41.041804 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:41.043325 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 466/947] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jul 18 10:16:40 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:16:41.102939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be945c96-c2d4-4712-bf52-5ab7ab93117c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:41.140811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d378a-6df4-4770-b6a9-2061262bc3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:41.141763 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d378a-6df4-4770-b6a9-2061262bc3cb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:41.142209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d378a-6df4-4770-b6a9-2061262bc3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:41.142644 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d378a-6df4-4770-b6a9-2061262bc3cb None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:41.143588 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d378a-6df4-4770-b6a9-2061262bc3cb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:41.144048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d378a-6df4-4770-b6a9-2061262bc3cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:41.145663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d378a-6df4-4770-b6a9-2061262bc3cb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:41.156008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 482/948] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 18 10:16:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:41.162328 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58dd60d-7ef7-4a7b-8cda-dfe1521f765d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:41.271767 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58dd60d-7ef7-4a7b-8cda-dfe1521f765d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:41.306437 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735fa6e-d078-4c57-bf4c-87ead2857178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:41.307095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735fa6e-d078-4c57-bf4c-87ead2857178 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:41.307380 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735fa6e-d078-4c57-bf4c-87ead2857178 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:41.307655 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735fa6e-d078-4c57-bf4c-87ead2857178 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:41.308410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735fa6e-d078-4c57-bf4c-87ead2857178 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:41.309811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735fa6e-d078-4c57-bf4c-87ead2857178 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:41.332374 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 467/949] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jul 18 10:16:41 2022] POST /identity/v3/endpoints => generated 347 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:16:42.423451 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62fe90-a735-49f0-b0e6-c90e3cbdf12a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:42.423965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62fe90-a735-49f0-b0e6-c90e3cbdf12a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:42.423965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62fe90-a735-49f0-b0e6-c90e3cbdf12a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:42.425559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 483/950] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:42.433330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b90b5-8cf4-45e2-9fb2-633c40831e69 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:42.433330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b90b5-8cf4-45e2-9fb2-633c40831e69 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:42.433330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b90b5-8cf4-45e2-9fb2-633c40831e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:42.450168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-aa0b90b5-8cf4-45e2-9fb2-633c40831e69 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:42.451681 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-aa0b90b5-8cf4-45e2-9fb2-633c40831e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:42.476713 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:42.537610 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 468/951] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:42.566263 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62fe90-a735-49f0-b0e6-c90e3cbdf12a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:42.672057 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62fe90-a735-49f0-b0e6-c90e3cbdf12a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:42.685497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:42.707826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bc06b-4b35-45b0-8bd6-9dfda84a78f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:42.708673 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bc06b-4b35-45b0-8bd6-9dfda84a78f6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:42.708961 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bc06b-4b35-45b0-8bd6-9dfda84a78f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:42.709242 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bc06b-4b35-45b0-8bd6-9dfda84a78f6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:42.720268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bc06b-4b35-45b0-8bd6-9dfda84a78f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:42.728833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-6b0bc06b-4b35-45b0-8bd6-9dfda84a78f6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:16:42.728833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:42.728833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:42.728833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:42.728833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:42.728833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:42.728833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:42.728833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:42.728833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:42.728833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:42.728833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:42.728833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:42.728833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:42.728833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:42.728833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:42.728833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:42.729784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:42.729784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:42.729784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:42.729784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:42.729784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:42.729784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:42.729784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:42.729784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:42.729784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:42.729784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:42.729784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:42.729784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:42.729784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:42.729784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:42.729784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:42.729784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:42.730915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:42.730915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:42.730915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:42.730915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:42.730915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:42.730915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:42.730915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:42.730915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:42.730915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:42.730915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:42.730915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:16:42.730915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:42.735205 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 484/952] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Jul 18 10:16:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:42.737615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b90b5-8cf4-45e2-9fb2-633c40831e69 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:42.849520 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b90b5-8cf4-45e2-9fb2-633c40831e69 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:42.853815 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283d2be-c007-4f3b-8e1f-be154265626b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:42.854086 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283d2be-c007-4f3b-8e1f-be154265626b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:42.854371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283d2be-c007-4f3b-8e1f-be154265626b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:42.856372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 485/953] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:42.862666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb97640-aa93-4750-bc19-7cf8e8d1d582 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:42.862956 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb97640-aa93-4750-bc19-7cf8e8d1d582 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:42.863240 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb97640-aa93-4750-bc19-7cf8e8d1d582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:42.885978 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-afb97640-aa93-4750-bc19-7cf8e8d1d582 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:42.887188 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-afb97640-aa93-4750-bc19-7cf8e8d1d582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:42.889979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a29c80-085a-4414-bd85-2026c445f1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:42.890890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a29c80-085a-4414-bd85-2026c445f1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:42.891366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a29c80-085a-4414-bd85-2026c445f1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:42.891818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a29c80-085a-4414-bd85-2026c445f1b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:42.892778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a29c80-085a-4414-bd85-2026c445f1b5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:42.893282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a29c80-085a-4414-bd85-2026c445f1b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:42.894843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a29c80-085a-4414-bd85-2026c445f1b5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:42.908320 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 469/954] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jul 18 10:16:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:42.909826 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a29c80-085a-4414-bd85-2026c445f1b5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:42.929589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:42.963840 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 486/955] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:16:42.988270 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb97640-aa93-4750-bc19-7cf8e8d1d582 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:43.026489 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a29c80-085a-4414-bd85-2026c445f1b5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:43.076419 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace1ffd-f5ad-4c96-bab2-7115b949d8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:43.078255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace1ffd-f5ad-4c96-bab2-7115b949d8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:43.078743 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace1ffd-f5ad-4c96-bab2-7115b949d8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:43.079228 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace1ffd-f5ad-4c96-bab2-7115b949d8d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:43.096075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace1ffd-f5ad-4c96-bab2-7115b949d8d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:43.098459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace1ffd-f5ad-4c96-bab2-7115b949d8d8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:43.102991 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb97640-aa93-4750-bc19-7cf8e8d1d582 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:43.108629 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:43.121981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4ace1ffd-f5ad-4c96-bab2-7115b949d8d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:43.121981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:43.121981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:43.121981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:43.121981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:43.121981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:43.121981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:43.121981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:43.121981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:43.121981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:43.121981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:43.121981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:43.121981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:43.121981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:43.121981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:43.123232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:43.123232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:43.123232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:43.123232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:43.123232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:43.123232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:43.123232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:43.123232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:43.123232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:43.123232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:43.123232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:43.123232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:43.123232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:43.123232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:43.123232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:43.123232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:43.123232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:43.124552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:43.124552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:43.124552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:43.124552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:43.126924 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 470/956] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:43.133109 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace1ffd-f5ad-4c96-bab2-7115b949d8d8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:43.151321 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7fece-570d-485b-a51e-af64a04b7e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:43.152091 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7fece-570d-485b-a51e-af64a04b7e3e None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:43.152491 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7fece-570d-485b-a51e-af64a04b7e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:43.152877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7fece-570d-485b-a51e-af64a04b7e3e None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:43.153785 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7fece-570d-485b-a51e-af64a04b7e3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:43.155446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7fece-570d-485b-a51e-af64a04b7e3e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:43.156889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: WARNING keystone.server.flask.common [None req-d2a7fece-570d-485b-a51e-af64a04b7e3e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 18 10:16:43.170365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.sql.core [None req-d2a7fece-570d-485b-a51e-af64a04b7e3e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 10:16:43.170365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 10:16:43.170365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [parameters: {'user_id': '6c0bc3d616a64f8c9fbcd480d33be8c1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 10:16:43.170365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80143) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 10:16:43.188945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d2a7fece-570d-485b-a51e-af64a04b7e3e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 10:16:43.188945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:43.188945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 10:16:43.188945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 10:16:43.188945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 10:16:43.188945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 10:16:43.188945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 10:16:43.188945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 10:16:43.188945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 10:16:43.188945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application conn.query(q) Jul 18 10:16:43.188945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 10:16:43.188945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 10:16:43.188945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 10:16:43.188945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result.read() Jul 18 10:16:43.188945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 10:16:43.189913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 10:16:43.189913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 10:16:43.189913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 10:16:43.189913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 10:16:43.189913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 10:16:43.189913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 10:16:43.189913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 10:16:43.189913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 10:16:43.189913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:43.189913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 10:16:43.189913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:43.189913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:43.189913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 10:16:43.189913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 10:16:43.189913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 18 10:16:43.189913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 18 10:16:43.189913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 10:16:43.189913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application next(self.gen) Jul 18 10:16:43.190917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 10:16:43.190917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application yield resource Jul 18 10:16:43.190917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 10:16:43.190917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application next(self.gen) Jul 18 10:16:43.190917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 10:16:43.190917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 10:16:43.190917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 10:16:43.190917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 10:16:43.190917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 10:16:43.190917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise self.value Jul 18 10:16:43.190917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 10:16:43.190917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 10:16:43.190917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 10:16:43.190917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application session.commit() Jul 18 10:16:43.190917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 10:16:43.190917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 10:16:43.190917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 10:16:43.192051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 10:16:43.192051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 10:16:43.192051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.session.flush() Jul 18 10:16:43.192051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 10:16:43.192051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 10:16:43.192051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 10:16:43.192051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 10:16:43.192051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 10:16:43.192051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application compat.raise_( Jul 18 10:16:43.192051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 10:16:43.192051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception Jul 18 10:16:43.192051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 10:16:43.192051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 10:16:43.192051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 10:16:43.192051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 10:16:43.192051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 10:16:43.192051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 10:16:43.194915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 10:16:43.194915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 10:16:43.194915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 18 10:16:43.194915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 18 10:16:43.194915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 10:16:43.194915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 10:16:43.194915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 10:16:43.194915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 10:16:43.194915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 10:16:43.194915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 10:16:43.194915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 10:16:43.194915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 10:16:43.194915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 10:16:43.194915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 10:16:43.194915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 10:16:43.194915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception Jul 18 10:16:43.197042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 10:16:43.197042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 10:16:43.197042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 10:16:43.197042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 10:16:43.197042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 10:16:43.197042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 10:16:43.197042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 10:16:43.197042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application conn.query(q) Jul 18 10:16:43.197042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 10:16:43.197042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 10:16:43.197042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 10:16:43.197042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result.read() Jul 18 10:16:43.197042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 10:16:43.197042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 10:16:43.197042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 10:16:43.197042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 10:16:43.199072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 10:16:43.199072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 10:16:43.199072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 10:16:43.199072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 10:16:43.199072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 10:16:43.199072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 10:16:43.199072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [parameters: {'user_id': '6c0bc3d616a64f8c9fbcd480d33be8c1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 10:16:43.199072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 10:16:43.199072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:43.199072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:43.199072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:43.199072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:43.199072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:43.199072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:43.199072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:43.201194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:43.201194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:43.201194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:43.201194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:43.201194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:43.201194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:43.201194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:43.201194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 18 10:16:43.201194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 18 10:16:43.201194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:43.201194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:43.201194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:43.201194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:43.201194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:43.201194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:43.201194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 18 10:16:43.201194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 18 10:16:43.204609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 18 10:16:43.204609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 18 10:16:43.204609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 10:16:43.204609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 10:16:43.204609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 10:16:43.204609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:43.204609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 487/957] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Jul 18 10:16:42 2022] POST /identity/v3/users => generated 160 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:43.204609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7fece-570d-485b-a51e-af64a04b7e3e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:43.225927 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:43.264413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace1ffd-f5ad-4c96-bab2-7115b949d8d8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:43.300366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81210b6b-029b-481a-b912-1d391a8ad1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:43.301113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81210b6b-029b-481a-b912-1d391a8ad1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:43.301569 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81210b6b-029b-481a-b912-1d391a8ad1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:43.301751 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81210b6b-029b-481a-b912-1d391a8ad1a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:43.302484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81210b6b-029b-481a-b912-1d391a8ad1a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:43.302796 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81210b6b-029b-481a-b912-1d391a8ad1a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:43.304298 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81210b6b-029b-481a-b912-1d391a8ad1a3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:43.316328 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 471/958] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:43.317684 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fb59a-261c-48a6-ae15-092a9dc9452a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:43.317974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fb59a-261c-48a6-ae15-092a9dc9452a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:43.318253 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fb59a-261c-48a6-ae15-092a9dc9452a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:43.320178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 472/959] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:43.325580 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057d6be-0763-4d70-a577-251261ac97eb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:43.325857 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057d6be-0763-4d70-a577-251261ac97eb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:43.326133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057d6be-0763-4d70-a577-251261ac97eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:43.327418 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7fece-570d-485b-a51e-af64a04b7e3e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:43.343292 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b057d6be-0763-4d70-a577-251261ac97eb None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:43.344478 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b057d6be-0763-4d70-a577-251261ac97eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:43.364965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645cb3d-2387-4653-8a58-f48c81dd0ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:43.365803 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645cb3d-2387-4653-8a58-f48c81dd0ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:43.366086 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645cb3d-2387-4653-8a58-f48c81dd0ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:43.366822 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645cb3d-2387-4653-8a58-f48c81dd0ae7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:43.377125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645cb3d-2387-4653-8a58-f48c81dd0ae7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:43.378943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645cb3d-2387-4653-8a58-f48c81dd0ae7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:43.394048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-6645cb3d-2387-4653-8a58-f48c81dd0ae7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:16:43.394048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:43.394048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:43.394048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:43.394048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:43.394048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:43.394048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:43.394048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:43.394048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:43.394048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:43.394048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:43.394048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:43.394048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:43.394048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:43.394048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:43.394048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:43.395229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:43.395229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:43.395229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:43.395229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:43.395229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:43.395229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:43.395229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:43.395229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:43.395229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:43.395229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:43.395229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:43.395229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:43.395229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:43.395229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:43.395229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:43.395229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:43.395229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:43.395229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:43.396171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:43.396171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:43.396171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:43.396171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:43.396171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:43.396171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:43.396171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:43.396171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:43.396171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 10:16:43.396171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:43.396171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 488/960] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:16:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:43.396171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645cb3d-2387-4653-8a58-f48c81dd0ae7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:43.419995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 473/961] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:43.421079 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057d6be-0763-4d70-a577-251261ac97eb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:43.429355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:43.484353 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:43.507301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645cb3d-2387-4653-8a58-f48c81dd0ae7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:43.525169 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057d6be-0763-4d70-a577-251261ac97eb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:43.539715 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff89f2-5eaa-4ba8-8a2a-1f8d9f785d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:43.540530 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff89f2-5eaa-4ba8-8a2a-1f8d9f785d5e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:43.540827 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff89f2-5eaa-4ba8-8a2a-1f8d9f785d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:43.541138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff89f2-5eaa-4ba8-8a2a-1f8d9f785d5e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:43.548328 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff89f2-5eaa-4ba8-8a2a-1f8d9f785d5e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:43.549780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff89f2-5eaa-4ba8-8a2a-1f8d9f785d5e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:43.558449 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67437a3c-4d6a-4eb8-85ec-baf9c1a53e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:43.559118 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67437a3c-4d6a-4eb8-85ec-baf9c1a53e76 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:43.559406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67437a3c-4d6a-4eb8-85ec-baf9c1a53e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:43.559685 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67437a3c-4d6a-4eb8-85ec-baf9c1a53e76 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:43.560492 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67437a3c-4d6a-4eb8-85ec-baf9c1a53e76 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:43.560921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67437a3c-4d6a-4eb8-85ec-baf9c1a53e76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:43.562367 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67437a3c-4d6a-4eb8-85ec-baf9c1a53e76 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:43.564415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-6dff89f2-5eaa-4ba8-8a2a-1f8d9f785d5e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:16:43.564415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:43.564415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:43.564415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:43.564415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:43.564415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:43.564415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:43.564415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:43.564415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:43.564415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:43.564415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:43.564415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:43.564415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:43.564415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:43.564415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:43.564415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:43.565548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:43.565548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:43.565548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:43.565548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:43.565548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:43.565548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:43.565548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:43.565548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:43.565548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:43.565548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:43.565548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:43.565548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:43.565548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:43.565548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:43.565548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:43.565548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:43.565548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:43.565548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:43.566709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:43.566709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:43.566709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:43.566709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:43.566709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:43.566709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:43.566709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:43.566709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:43.566709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 10:16:43.566709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:43.566709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 489/962] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 18 10:16:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:43.567459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f6fee-c68f-40b1-99e3-923b7c13f028 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:43.567585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f6fee-c68f-40b1-99e3-923b7c13f028 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:43.567862 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f6fee-c68f-40b1-99e3-923b7c13f028 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:43.571955 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 490/963] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:43.572679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f6fee-c68f-40b1-99e3-923b7c13f028 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:43.601931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 474/964] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jul 18 10:16:43 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:43.602769 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67437a3c-4d6a-4eb8-85ec-baf9c1a53e76 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:43.671766 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:43.695649 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f6fee-c68f-40b1-99e3-923b7c13f028 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:43.715378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67437a3c-4d6a-4eb8-85ec-baf9c1a53e76 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:43.735277 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5dbfd1-e921-4c29-9518-9c7d6c31c51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:43.735585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:43.736278 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5dbfd1-e921-4c29-9518-9c7d6c31c51f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:43.736850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5dbfd1-e921-4c29-9518-9c7d6c31c51f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:43.737260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5dbfd1-e921-4c29-9518-9c7d6c31c51f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:43.738174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5dbfd1-e921-4c29-9518-9c7d6c31c51f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:43.739804 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5dbfd1-e921-4c29-9518-9c7d6c31c51f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:43.753072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0f5dbfd1-e921-4c29-9518-9c7d6c31c51f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:16:43.753072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:43.753072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:43.753072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:43.753072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:43.753072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:43.753072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:43.753072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:43.753072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:16:43.753072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:43.753072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:43.753072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:43.753072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:43.753072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:43.753072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:43.753072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:43.753072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:43.754423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:43.754423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:43.754423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:43.754423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:43.754423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:43.754423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:43.754423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:43.754423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:43.754423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:43.754423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:43.754423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:43.754423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:43.754423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:43.754423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:43.754423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:43.754423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:43.755665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:43.755665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:43.755665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:43.755665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:43.755665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:43.755665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:43.755665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:43.755665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:43.755665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:43.755665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:43.755665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:43.755665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:43.755665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:16:43.755665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:43.755665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 491/965] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 18 10:16:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:43.758402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce56f6-6dd9-4e57-92a7-d29e5f935db6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:43.758402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce56f6-6dd9-4e57-92a7-d29e5f935db6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:43.758402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce56f6-6dd9-4e57-92a7-d29e5f935db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:43.760537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916dd9d0-0bd0-4ca7-a694-e21e334e6900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:43.761312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916dd9d0-0bd0-4ca7-a694-e21e334e6900 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:43.761697 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916dd9d0-0bd0-4ca7-a694-e21e334e6900 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:43.762079 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916dd9d0-0bd0-4ca7-a694-e21e334e6900 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:43.763020 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916dd9d0-0bd0-4ca7-a694-e21e334e6900 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:43.763413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916dd9d0-0bd0-4ca7-a694-e21e334e6900 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:43.765140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916dd9d0-0bd0-4ca7-a694-e21e334e6900 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:43.777928 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d2ce56f6-6dd9-4e57-92a7-d29e5f935db6 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:43.779433 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d2ce56f6-6dd9-4e57-92a7-d29e5f935db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:43.784236 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 475/966] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Jul 18 10:16:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:43.785246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916dd9d0-0bd0-4ca7-a694-e21e334e6900 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:43.858329 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 492/967] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:43.859585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce56f6-6dd9-4e57-92a7-d29e5f935db6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:43.904854 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916dd9d0-0bd0-4ca7-a694-e21e334e6900 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:43.941767 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6a1f6-e59f-4b68-b8cc-648369c43384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:43.942423 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6a1f6-e59f-4b68-b8cc-648369c43384 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:43.942703 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6a1f6-e59f-4b68-b8cc-648369c43384 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:43.942982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6a1f6-e59f-4b68-b8cc-648369c43384 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:43.943686 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6a1f6-e59f-4b68-b8cc-648369c43384 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:43.944019 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6a1f6-e59f-4b68-b8cc-648369c43384 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:43.944426 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:43.945630 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6a1f6-e59f-4b68-b8cc-648369c43384 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:43.950440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 476/968] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 18 10:16:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:43.951215 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6a1f6-e59f-4b68-b8cc-648369c43384 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:43.971534 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce56f6-6dd9-4e57-92a7-d29e5f935db6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:43.992051 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:44.007749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf3fa5-7fb7-43c2-8479-a3b860d51448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:44.008406 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf3fa5-7fb7-43c2-8479-a3b860d51448 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:44.008733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf3fa5-7fb7-43c2-8479-a3b860d51448 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:44.009016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf3fa5-7fb7-43c2-8479-a3b860d51448 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:44.048429 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6a1f6-e59f-4b68-b8cc-648369c43384 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:44.077336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf3fa5-7fb7-43c2-8479-a3b860d51448 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:44.078906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf3fa5-7fb7-43c2-8479-a3b860d51448 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:44.081500 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155676f6-421d-493a-b7d4-cfa1d91722ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:44.082284 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155676f6-421d-493a-b7d4-cfa1d91722ae None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:44.082566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155676f6-421d-493a-b7d4-cfa1d91722ae None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:44.082841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155676f6-421d-493a-b7d4-cfa1d91722ae None admin] PATH_INFO: `/v3/services/compute` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:44.083486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155676f6-421d-493a-b7d4-cfa1d91722ae None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:44.084867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155676f6-421d-493a-b7d4-cfa1d91722ae None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:44.096206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-155676f6-421d-493a-b7d4-cfa1d91722ae None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:16:44.096206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:44.096206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:44.096206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:44.096206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:44.096206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:44.096206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:44.096206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:44.096206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:16:44.096206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:44.096206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:44.096206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:44.096206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:44.096206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:44.096206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:44.096206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:44.096206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:44.097460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:44.097460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:44.097460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:44.097460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:44.097460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:44.097460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:44.097460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:44.097460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:44.097460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:44.097460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:44.097460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:44.097460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:44.097460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:44.097460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:44.097460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:44.097460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:44.098312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:44.098312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:44.098312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:44.098312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:44.098312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:44.098312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:44.098312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:44.098312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:44.098312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:44.098312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:44.098312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:44.098312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:44.098312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:16:44.098312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:44.098312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 477/969] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:44.099103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155676f6-421d-493a-b7d4-cfa1d91722ae None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:44.159891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:44.185750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 493/970] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:16:43 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:16:44.186598 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf3fa5-7fb7-43c2-8479-a3b860d51448 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:44.204020 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155676f6-421d-493a-b7d4-cfa1d91722ae None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:44.206899 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:44.238213 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a3e7f-5fd3-4b19-b9e0-80a6be05dc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:44.238874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a3e7f-5fd3-4b19-b9e0-80a6be05dc6c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:44.239157 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a3e7f-5fd3-4b19-b9e0-80a6be05dc6c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:44.239435 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a3e7f-5fd3-4b19-b9e0-80a6be05dc6c None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:44.240141 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a3e7f-5fd3-4b19-b9e0-80a6be05dc6c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:44.240449 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a3e7f-5fd3-4b19-b9e0-80a6be05dc6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:44.242470 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a3e7f-5fd3-4b19-b9e0-80a6be05dc6c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:44.247158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 478/971] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 18 10:16:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:44.247936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a3e7f-5fd3-4b19-b9e0-80a6be05dc6c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:44.292887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf3fa5-7fb7-43c2-8479-a3b860d51448 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:44.326401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351c236-ffd4-42f9-aa96-43596d206e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:44.327059 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351c236-ffd4-42f9-aa96-43596d206e2e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:44.327346 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351c236-ffd4-42f9-aa96-43596d206e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:44.327630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351c236-ffd4-42f9-aa96-43596d206e2e None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:44.328343 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351c236-ffd4-42f9-aa96-43596d206e2e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:44.328688 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351c236-ffd4-42f9-aa96-43596d206e2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:44.330085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6351c236-ffd4-42f9-aa96-43596d206e2e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:44.334799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 494/972] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 18 10:16:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:44.335712 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351c236-ffd4-42f9-aa96-43596d206e2e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:44.345519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a3e7f-5fd3-4b19-b9e0-80a6be05dc6c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:44.378543 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db0eaa-cac4-44ae-a6b9-4241a3ea8ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:44.379356 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db0eaa-cac4-44ae-a6b9-4241a3ea8ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:44.379614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db0eaa-cac4-44ae-a6b9-4241a3ea8ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:44.379890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db0eaa-cac4-44ae-a6b9-4241a3ea8ba2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:44.385209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:44.391177 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db0eaa-cac4-44ae-a6b9-4241a3ea8ba2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:44.392654 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db0eaa-cac4-44ae-a6b9-4241a3ea8ba2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:44.409455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-55db0eaa-cac4-44ae-a6b9-4241a3ea8ba2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:44.409455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:44.409455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:44.409455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:44.409455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:44.409455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:44.409455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:44.409455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:44.409455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:44.409455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:44.409455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:44.409455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:44.409455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:44.409455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:44.409455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:44.410532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:44.410532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:44.410532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:44.410532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:44.410532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:44.410532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:44.410532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:44.410532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:44.410532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:44.410532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:44.410532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:44.410532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:44.410532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:44.410532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:44.410532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:44.410532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:44.410532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:44.416241 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:44.416241 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:44.416241 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:44.416241 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:44.416241 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 479/973] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:44.416241 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db0eaa-cac4-44ae-a6b9-4241a3ea8ba2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:44.432979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:44.447370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351c236-ffd4-42f9-aa96-43596d206e2e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:44.480091 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abbac0-7f62-4fb4-8df3-814d0aaf1d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:44.481045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abbac0-7f62-4fb4-8df3-814d0aaf1d2c None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:44.481332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abbac0-7f62-4fb4-8df3-814d0aaf1d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:44.481623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abbac0-7f62-4fb4-8df3-814d0aaf1d2c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:44.482352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6abbac0-7f62-4fb4-8df3-814d0aaf1d2c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:44.483720 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6abbac0-7f62-4fb4-8df3-814d0aaf1d2c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:44.502151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 495/974] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jul 18 10:16:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:16:44.502889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abbac0-7f62-4fb4-8df3-814d0aaf1d2c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:44.513807 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db0eaa-cac4-44ae-a6b9-4241a3ea8ba2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:44.564402 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c7edd-dd85-4506-9024-5281ac22e918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:44.567866 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c7edd-dd85-4506-9024-5281ac22e918 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:44.568497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c7edd-dd85-4506-9024-5281ac22e918 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:44.568913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c7edd-dd85-4506-9024-5281ac22e918 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:44.569814 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c7edd-dd85-4506-9024-5281ac22e918 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:44.570144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c7edd-dd85-4506-9024-5281ac22e918 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:44.572090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c7edd-dd85-4506-9024-5281ac22e918 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:44.581265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 480/975] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 18 10:16:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:44.659970 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:44.666552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abbac0-7f62-4fb4-8df3-814d0aaf1d2c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:44.716019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5255f6-6165-43a9-b3e4-1bbfddb192f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:44.716853 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5255f6-6165-43a9-b3e4-1bbfddb192f5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:44.717207 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5255f6-6165-43a9-b3e4-1bbfddb192f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:44.717550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5255f6-6165-43a9-b3e4-1bbfddb192f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:44.718369 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5255f6-6165-43a9-b3e4-1bbfddb192f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:44.718751 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5255f6-6165-43a9-b3e4-1bbfddb192f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:44.720213 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5255f6-6165-43a9-b3e4-1bbfddb192f5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:44.738671 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 496/976] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:44.752733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c7edd-dd85-4506-9024-5281ac22e918 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:44.856813 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:44.869404 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c7edd-dd85-4506-9024-5281ac22e918 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:44.903638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d4a81-6cce-425a-8c58-39e0b1096dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:44.904441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d4a81-6cce-425a-8c58-39e0b1096dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:44.904762 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d4a81-6cce-425a-8c58-39e0b1096dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:44.905037 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d4a81-6cce-425a-8c58-39e0b1096dd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:44.912353 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d4a81-6cce-425a-8c58-39e0b1096dd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:44.913912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d4a81-6cce-425a-8c58-39e0b1096dd1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:44.928586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ef2d4a81-6cce-425a-8c58-39e0b1096dd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:44.928586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:44.928586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:44.928586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:44.928586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:44.928586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:44.928586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:44.928586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:44.928586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:44.928586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:44.928586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:44.928586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:44.928586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:44.928586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:44.928586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:44.929775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:44.929775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:44.929775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:44.929775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:44.929775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:44.929775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:44.929775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:44.929775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:44.929775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:44.929775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:44.929775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:44.929775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:44.929775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:44.929775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:44.929775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:44.929775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:44.929775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:44.930675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:44.930675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:44.930675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:44.930675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:44.930675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:44.930675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:44.930675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:44.930675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:44.935793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5255f6-6165-43a9-b3e4-1bbfddb192f5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:44.936821 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 481/977] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:45.043498 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5255f6-6165-43a9-b3e4-1bbfddb192f5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:45.084212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3782e0-f242-421d-8b54-5c233aaf5ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:45.085713 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3782e0-f242-421d-8b54-5c233aaf5ffc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:45.086001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3782e0-f242-421d-8b54-5c233aaf5ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:45.086280 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3782e0-f242-421d-8b54-5c233aaf5ffc None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:45.087078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3782e0-f242-421d-8b54-5c233aaf5ffc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:45.087396 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3782e0-f242-421d-8b54-5c233aaf5ffc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:45.088887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3782e0-f242-421d-8b54-5c233aaf5ffc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:45.103693 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 497/978] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:45.110136 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d4a81-6cce-425a-8c58-39e0b1096dd1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:45.217045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d4a81-6cce-425a-8c58-39e0b1096dd1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:45.252304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e77569-00fe-4eed-b83b-c34752c11cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:45.253013 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e77569-00fe-4eed-b83b-c34752c11cfb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:45.253295 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e77569-00fe-4eed-b83b-c34752c11cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:45.253541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e77569-00fe-4eed-b83b-c34752c11cfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:45.254372 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e77569-00fe-4eed-b83b-c34752c11cfb None admin] RBAC: Adding query filter params (role.id=25bb48ece7784b5d9b0a60db78e81edd, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=a09029144c454b099bea86aaa5229ef0) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:45.254686 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e77569-00fe-4eed-b83b-c34752c11cfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:45.256116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e77569-00fe-4eed-b83b-c34752c11cfb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:45.268582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 482/979] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:16:45 2022] GET /identity/v3/role_assignments?user.id=a09029144c454b099bea86aaa5229ef0&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=25bb48ece7784b5d9b0a60db78e81edd => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:46.609805 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c6021-6064-4bb4-bd31-5ebe8aa335bb None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:46.610401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c6021-6064-4bb4-bd31-5ebe8aa335bb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:46.610401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c6021-6064-4bb4-bd31-5ebe8aa335bb None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:46.611825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 498/980] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:46.612476 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7dec0c-488d-4a14-a692-4e731c07fe5c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:46.612996 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7dec0c-488d-4a14-a692-4e731c07fe5c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:46.613074 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7dec0c-488d-4a14-a692-4e731c07fe5c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:46.614678 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 483/981] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:16:46.616920 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5a3cd-fea7-4eda-9cd4-e64c428c2792 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:46.617213 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5a3cd-fea7-4eda-9cd4-e64c428c2792 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:46.617495 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5a3cd-fea7-4eda-9cd4-e64c428c2792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:46.619661 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77264928-8776-4d5f-a697-f74b3ced7145 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:46.619949 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77264928-8776-4d5f-a697-f74b3ced7145 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:46.620223 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77264928-8776-4d5f-a697-f74b3ced7145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:46.626830 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:46.635505 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8cb5a3cd-fea7-4eda-9cd4-e64c428c2792 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:46.636728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8cb5a3cd-fea7-4eda-9cd4-e64c428c2792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:46.638365 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-77264928-8776-4d5f-a697-f74b3ced7145 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:46.639512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-77264928-8776-4d5f-a697-f74b3ced7145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:46.650733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:46.712547 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 499/982] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:46.712765 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 484/983] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:16:46.731050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5a3cd-fea7-4eda-9cd4-e64c428c2792 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:46.731735 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77264928-8776-4d5f-a697-f74b3ced7145 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:46.835682 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77264928-8776-4d5f-a697-f74b3ced7145 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:46.839118 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5a3cd-fea7-4eda-9cd4-e64c428c2792 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:46.878804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ed139-e64a-4ec7-9804-c2d776c6b141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:46.879020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a1e2b-8fba-4c6b-a695-b66cfa823287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:46.879477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ed139-e64a-4ec7-9804-c2d776c6b141 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:46.879671 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a1e2b-8fba-4c6b-a695-b66cfa823287 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:46.879797 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ed139-e64a-4ec7-9804-c2d776c6b141 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:46.879951 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a1e2b-8fba-4c6b-a695-b66cfa823287 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:46.880105 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ed139-e64a-4ec7-9804-c2d776c6b141 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:46.880198 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a1e2b-8fba-4c6b-a695-b66cfa823287 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:46.880850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ed139-e64a-4ec7-9804-c2d776c6b141 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:46.880971 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a1e2b-8fba-4c6b-a695-b66cfa823287 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:46.882842 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ed139-e64a-4ec7-9804-c2d776c6b141 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:46.882944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a1e2b-8fba-4c6b-a695-b66cfa823287 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:46.885203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:46.896988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-cd4a1e2b-8fba-4c6b-a695-b66cfa823287 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:16:46.896988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:46.896988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:46.896988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:46.896988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:46.896988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:46.896988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:46.896988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:46.896988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:16:46.896988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:46.896988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:46.896988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:46.896988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:46.896988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:46.896988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:46.896988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:46.896988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:46.898070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:46.898070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:46.898070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:46.898070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:46.898070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:46.898070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:46.898070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:46.898070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:46.898070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:46.898070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:46.898070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:46.898070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:46.898070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:46.898070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:46.898070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:46.898070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:46.898973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:46.898973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:46.898973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:46.898973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:46.898973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:46.898973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:46.898973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:46.898973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:46.898973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:46.898973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:46.898973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:46.898973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:46.898973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:16:46.898973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:46.898973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 500/984] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 18 10:16:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:46.904580 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 485/985] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 18 10:16:46 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:16:46.904735 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a1e2b-8fba-4c6b-a695-b66cfa823287 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:46.924005 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:47.022832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a1e2b-8fba-4c6b-a695-b66cfa823287 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:47.066408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f05ca-5262-436a-9a80-f4776bd946a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:47.067073 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f05ca-5262-436a-9a80-f4776bd946a4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:47.067356 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f05ca-5262-436a-9a80-f4776bd946a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:47.067638 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f05ca-5262-436a-9a80-f4776bd946a4 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:47.068339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f05ca-5262-436a-9a80-f4776bd946a4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:47.068667 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f05ca-5262-436a-9a80-f4776bd946a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:47.070123 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f05ca-5262-436a-9a80-f4776bd946a4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:47.075506 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 501/986] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 18 10:16:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:47.081426 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ed139-e64a-4ec7-9804-c2d776c6b141 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:47.192339 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ed139-e64a-4ec7-9804-c2d776c6b141 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:47.227203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e5729-12f3-473b-9769-df22f7f822f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:47.227872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e5729-12f3-473b-9769-df22f7f822f5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:47.228159 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e5729-12f3-473b-9769-df22f7f822f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:47.228437 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e5729-12f3-473b-9769-df22f7f822f5 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:47.229179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e5729-12f3-473b-9769-df22f7f822f5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:47.229492 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e5729-12f3-473b-9769-df22f7f822f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:47.231123 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e5729-12f3-473b-9769-df22f7f822f5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:47.236393 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 486/987] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 18 10:16:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:47.242400 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f05ca-5262-436a-9a80-f4776bd946a4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:47.305088 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ae4f3-904d-4f89-8cb7-eaa927ae4503 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:47.305379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ae4f3-904d-4f89-8cb7-eaa927ae4503 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:47.305662 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ae4f3-904d-4f89-8cb7-eaa927ae4503 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:47.307169 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 487/988] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:16:47.311963 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab3cb1-0b97-461f-ae22-a148087ff94f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:47.312232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab3cb1-0b97-461f-ae22-a148087ff94f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:47.312537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab3cb1-0b97-461f-ae22-a148087ff94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:47.318503 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:47.330635 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a6ab3cb1-0b97-461f-ae22-a148087ff94f None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:47.331826 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a6ab3cb1-0b97-461f-ae22-a148087ff94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:47.349938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f05ca-5262-436a-9a80-f4776bd946a4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:47.384653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3f06c-56fa-45b5-a0b6-973b584d824c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:47.385308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3f06c-56fa-45b5-a0b6-973b584d824c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:47.385592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3f06c-56fa-45b5-a0b6-973b584d824c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:47.385866 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3f06c-56fa-45b5-a0b6-973b584d824c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:47.386669 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3f06c-56fa-45b5-a0b6-973b584d824c None admin] RBAC: Adding query filter params (interface=admin, service_id=baa482f55c1e4bf491ab6f1092ff9f54, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:47.386810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:47.386974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3f06c-56fa-45b5-a0b6-973b584d824c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:47.388355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3f06c-56fa-45b5-a0b6-973b584d824c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:47.394094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 502/989] 10.222.0.33 () {56 vars in 1249 bytes} [Mon Jul 18 10:16:47 2022] GET /identity/v3/endpoints?service_id=baa482f55c1e4bf491ab6f1092ff9f54&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:47.400190 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3f06c-56fa-45b5-a0b6-973b584d824c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:47.416607 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 488/990] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:16:47.435234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab3cb1-0b97-461f-ae22-a148087ff94f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:47.510352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3f06c-56fa-45b5-a0b6-973b584d824c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:47.543966 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab3cb1-0b97-461f-ae22-a148087ff94f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:47.544477 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511e247-ea08-41b6-b011-3966c3420ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:47.545182 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511e247-ea08-41b6-b011-3966c3420ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:47.545580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511e247-ea08-41b6-b011-3966c3420ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:47.545856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511e247-ea08-41b6-b011-3966c3420ff3 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:47.546578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511e247-ea08-41b6-b011-3966c3420ff3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:47.547950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511e247-ea08-41b6-b011-3966c3420ff3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:47.549807 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:47.557261 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 503/991] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jul 18 10:16:47 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:16:47.582723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38332a-4f28-4760-965a-29739cd2c228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:47.583801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38332a-4f28-4760-965a-29739cd2c228 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:47.584266 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38332a-4f28-4760-965a-29739cd2c228 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:47.584750 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38332a-4f28-4760-965a-29739cd2c228 None admin] PATH_INFO: `/v3/services/image` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:47.585633 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38332a-4f28-4760-965a-29739cd2c228 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:47.587192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38332a-4f28-4760-965a-29739cd2c228 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:47.600158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4f38332a-4f28-4760-965a-29739cd2c228 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:16:47.600158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:47.600158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:47.600158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:47.600158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:47.600158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:47.600158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:47.600158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:47.600158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:16:47.600158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:47.600158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:47.600158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:47.600158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:47.600158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:47.600158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:47.600158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:47.600158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:47.601408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:47.601408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:47.601408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:47.601408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:47.601408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:47.601408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:47.601408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:47.601408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:47.601408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:47.601408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:47.601408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:47.601408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:47.601408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:47.601408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:47.601408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:47.601408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:47.602265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:47.602265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:47.602265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:47.602265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:47.602265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:47.602265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:47.602265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:47.602265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:47.602265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:47.602265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:47.602265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:47.602265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:47.602265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:16:47.602265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:47.603024 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 489/992] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Jul 18 10:16:47 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:47.609059 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511e247-ea08-41b6-b011-3966c3420ff3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:47.719080 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511e247-ea08-41b6-b011-3966c3420ff3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:47.754659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af213b4-fd44-48d4-8adc-5b34c2742513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:47.755326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af213b4-fd44-48d4-8adc-5b34c2742513 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:47.755609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af213b4-fd44-48d4-8adc-5b34c2742513 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:47.755885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af213b4-fd44-48d4-8adc-5b34c2742513 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:47.756661 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af213b4-fd44-48d4-8adc-5b34c2742513 None admin] RBAC: Adding query filter params (name=image) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:47.756975 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af213b4-fd44-48d4-8adc-5b34c2742513 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:47.758431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af213b4-fd44-48d4-8adc-5b34c2742513 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:47.763593 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 504/993] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:47.769548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38332a-4f28-4760-965a-29739cd2c228 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:47.825332 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:47.873745 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38332a-4f28-4760-965a-29739cd2c228 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:47.907149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da20bee-c1e2-46c2-883a-ed38b78fc940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:47.907867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da20bee-c1e2-46c2-883a-ed38b78fc940 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:47.908153 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da20bee-c1e2-46c2-883a-ed38b78fc940 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:47.908429 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da20bee-c1e2-46c2-883a-ed38b78fc940 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:47.909168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da20bee-c1e2-46c2-883a-ed38b78fc940 None admin] RBAC: Adding query filter params (type=image) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:47.909478 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da20bee-c1e2-46c2-883a-ed38b78fc940 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:47.910844 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da20bee-c1e2-46c2-883a-ed38b78fc940 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:47.924610 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 490/994] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:47 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:48.972816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd5a7a-d8e8-4896-b8ff-f685fe49d746 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:48.973405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd5a7a-d8e8-4896-b8ff-f685fe49d746 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:48.973516 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd5a7a-d8e8-4896-b8ff-f685fe49d746 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:48.975035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 505/995] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:48.980840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306fee8-7347-4aea-8018-01e247593eb6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:48.981234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306fee8-7347-4aea-8018-01e247593eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:48.981450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306fee8-7347-4aea-8018-01e247593eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:48.998298 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9306fee8-7347-4aea-8018-01e247593eb6 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:48.999497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9306fee8-7347-4aea-8018-01e247593eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:49.072859 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 491/996] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:49.095717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd5a7a-d8e8-4896-b8ff-f685fe49d746 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:49.163488 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:49.204927 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd5a7a-d8e8-4896-b8ff-f685fe49d746 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:49.239244 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31735253-42c8-42c8-b0ab-594eaef1410a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:49.239978 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31735253-42c8-42c8-b0ab-594eaef1410a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:49.240262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31735253-42c8-42c8-b0ab-594eaef1410a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:49.240564 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31735253-42c8-42c8-b0ab-594eaef1410a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:49.241244 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31735253-42c8-42c8-b0ab-594eaef1410a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:49.242679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31735253-42c8-42c8-b0ab-594eaef1410a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:49.254528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-31735253-42c8-42c8-b0ab-594eaef1410a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:16:49.254528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:49.254528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:49.254528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:49.254528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:49.254528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:49.254528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:49.254528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:49.254528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:16:49.254528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:49.254528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:49.254528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:49.254528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:49.254528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:49.254528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:49.254528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:49.254528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:49.255557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:49.255557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:49.255557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:49.255557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:49.255557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:49.255557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:49.255557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:49.255557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:49.255557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:49.255557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:49.255557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:49.255557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:49.255557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:49.255557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:49.255557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:49.255557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:49.261447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:49.261447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:49.261447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:49.261447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:49.261447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:49.261447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:49.261447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:49.261447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:49.261447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:49.261447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:49.261447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:49.261447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:49.261447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:16:49.261447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:49.261447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 506/997] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:49.263991 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9306fee8-7347-4aea-8018-01e247593eb6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:49.310477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:49.364243 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9306fee8-7347-4aea-8018-01e247593eb6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:49.398124 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56dcf6-f94c-4899-bb70-f955a1e467c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:49.398785 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56dcf6-f94c-4899-bb70-f955a1e467c8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:49.399066 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56dcf6-f94c-4899-bb70-f955a1e467c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:49.399336 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56dcf6-f94c-4899-bb70-f955a1e467c8 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:49.400035 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56dcf6-f94c-4899-bb70-f955a1e467c8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:49.400337 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56dcf6-f94c-4899-bb70-f955a1e467c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:49.401714 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56dcf6-f94c-4899-bb70-f955a1e467c8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:49.411199 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 492/998] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 18 10:16:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:49.412162 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31735253-42c8-42c8-b0ab-594eaef1410a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:49.511627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31735253-42c8-42c8-b0ab-594eaef1410a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:49.537671 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:49.546360 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b31bd-3e1c-4901-9cd0-b0fc93dd4502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:49.547020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b31bd-3e1c-4901-9cd0-b0fc93dd4502 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:49.547301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b31bd-3e1c-4901-9cd0-b0fc93dd4502 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:49.547574 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b31bd-3e1c-4901-9cd0-b0fc93dd4502 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:49.548304 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b31bd-3e1c-4901-9cd0-b0fc93dd4502 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:49.548610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b31bd-3e1c-4901-9cd0-b0fc93dd4502 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:49.549975 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b31bd-3e1c-4901-9cd0-b0fc93dd4502 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:49.560088 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 507/999] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 18 10:16:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:49.561094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b31bd-3e1c-4901-9cd0-b0fc93dd4502 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:49.561695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28de56d-14c8-4af5-a6c0-43101519fec7 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:49.561991 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28de56d-14c8-4af5-a6c0-43101519fec7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:49.562268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28de56d-14c8-4af5-a6c0-43101519fec7 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:49.563633 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 493/1000] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:49.568616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12031f1-915c-4709-9257-3437a5b18bfb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:49.569479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12031f1-915c-4709-9257-3437a5b18bfb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:49.569762 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12031f1-915c-4709-9257-3437a5b18bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:49.586141 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f12031f1-915c-4709-9257-3437a5b18bfb None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:49.587330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f12031f1-915c-4709-9257-3437a5b18bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:49.663559 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 494/1001] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:16:49.677268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b31bd-3e1c-4901-9cd0-b0fc93dd4502 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:49.686272 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12031f1-915c-4709-9257-3437a5b18bfb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:49.721274 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a9870-ba2d-42e8-9ca1-d4719dd4321b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:49.722180 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a9870-ba2d-42e8-9ca1-d4719dd4321b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:49.722628 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a9870-ba2d-42e8-9ca1-d4719dd4321b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:49.723086 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a9870-ba2d-42e8-9ca1-d4719dd4321b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:49.724031 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a9870-ba2d-42e8-9ca1-d4719dd4321b None admin] RBAC: Adding query filter params (interface=public, service_id=ad0ced5b09ca4f6e9f3d4fd6acaf59db, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:49.724916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a9870-ba2d-42e8-9ca1-d4719dd4321b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:49.726619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:49.726822 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a9870-ba2d-42e8-9ca1-d4719dd4321b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:49.734625 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 508/1002] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jul 18 10:16:49 2022] GET /identity/v3/endpoints?service_id=ad0ced5b09ca4f6e9f3d4fd6acaf59db&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:49.739939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a9870-ba2d-42e8-9ca1-d4719dd4321b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:49.791561 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12031f1-915c-4709-9257-3437a5b18bfb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:49.804485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:49.831562 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b27a78-f490-4f78-9240-eaf37cb709bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:49.832371 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b27a78-f490-4f78-9240-eaf37cb709bd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:49.832692 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b27a78-f490-4f78-9240-eaf37cb709bd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:49.832974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b27a78-f490-4f78-9240-eaf37cb709bd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:49.833751 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b27a78-f490-4f78-9240-eaf37cb709bd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:49.835283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b27a78-f490-4f78-9240-eaf37cb709bd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:49.848112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a1b27a78-f490-4f78-9240-eaf37cb709bd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:16:49.848112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:49.848112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:49.848112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:49.848112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:49.848112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:49.848112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:49.848112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:49.848112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:16:49.848112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:49.848112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:49.848112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:49.848112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:49.848112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:49.848112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:49.848112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:49.848112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:49.852545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:49.852545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:49.852545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:49.852545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:49.852545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:49.852545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:49.852545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:49.852545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:49.852545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:49.852545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:49.852545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:49.852545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:49.852545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:49.852545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:49.852545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:49.852545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:49.853501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:49.853501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:49.853501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:49.853501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:49.853501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:49.853501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:49.853501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:49.853501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:49.853501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:49.853501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:49.853501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:49.853501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:49.853501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 10:16:49.853501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:49.854345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a9870-ba2d-42e8-9ca1-d4719dd4321b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:49.854421 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 495/1003] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jul 18 10:16:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:49.855134 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b27a78-f490-4f78-9240-eaf37cb709bd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:49.892259 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b057f0-bcb4-419d-9e83-976cdbf2338f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:49.893698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b057f0-bcb4-419d-9e83-976cdbf2338f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:49.893988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b057f0-bcb4-419d-9e83-976cdbf2338f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:49.894269 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b057f0-bcb4-419d-9e83-976cdbf2338f None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:49.894950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b057f0-bcb4-419d-9e83-976cdbf2338f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:49.896310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b057f0-bcb4-419d-9e83-976cdbf2338f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:49.913015 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 509/1004] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jul 18 10:16:49 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:16:49.956074 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadd62c-24aa-400a-b96d-3dab47aa92b5 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:49.956369 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadd62c-24aa-400a-b96d-3dab47aa92b5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:49.956688 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadd62c-24aa-400a-b96d-3dab47aa92b5 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:49.958144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 510/1005] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:16:49.964113 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa9446-4319-4a00-8055-85751eb476d8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:49.964443 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa9446-4319-4a00-8055-85751eb476d8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:49.964709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa9446-4319-4a00-8055-85751eb476d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:49.979239 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b27a78-f490-4f78-9240-eaf37cb709bd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:49.982397 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:49.983025 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-15fa9446-4319-4a00-8055-85751eb476d8 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:49.984276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-15fa9446-4319-4a00-8055-85751eb476d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:50.024301 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49faee87-21ce-4fb6-bd26-3f5626412f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:50.025059 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49faee87-21ce-4fb6-bd26-3f5626412f96 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:50.025348 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49faee87-21ce-4fb6-bd26-3f5626412f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:50.025625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49faee87-21ce-4fb6-bd26-3f5626412f96 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:50.026651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49faee87-21ce-4fb6-bd26-3f5626412f96 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:50.026960 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49faee87-21ce-4fb6-bd26-3f5626412f96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:50.028345 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49faee87-21ce-4fb6-bd26-3f5626412f96 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:50.034091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 496/1006] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 18 10:16:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:50.040325 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49faee87-21ce-4fb6-bd26-3f5626412f96 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:50.071965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 511/1007] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:16:50.095145 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa9446-4319-4a00-8055-85751eb476d8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:50.149008 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49faee87-21ce-4fb6-bd26-3f5626412f96 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:50.157704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:50.189165 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b059483-5390-4a91-8b2f-ae5f73333113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:50.189937 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b059483-5390-4a91-8b2f-ae5f73333113 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:50.190230 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b059483-5390-4a91-8b2f-ae5f73333113 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:50.190512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b059483-5390-4a91-8b2f-ae5f73333113 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:50.191231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b059483-5390-4a91-8b2f-ae5f73333113 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:50.191549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b059483-5390-4a91-8b2f-ae5f73333113 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:50.193038 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b059483-5390-4a91-8b2f-ae5f73333113 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:50.199019 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 497/1008] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jul 18 10:16:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:50.204341 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b059483-5390-4a91-8b2f-ae5f73333113 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:50.215551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa9446-4319-4a00-8055-85751eb476d8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:50.236742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:50.253442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97686d-a17b-4201-8558-54bd07a6e860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:50.254110 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97686d-a17b-4201-8558-54bd07a6e860 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:50.254462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97686d-a17b-4201-8558-54bd07a6e860 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:50.254743 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97686d-a17b-4201-8558-54bd07a6e860 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:50.255467 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97686d-a17b-4201-8558-54bd07a6e860 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:50.257416 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97686d-a17b-4201-8558-54bd07a6e860 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:50.271541 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 512/1009] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 18 10:16:50 2022] POST /identity/v3/services => generated 242 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:16:50.320858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b059483-5390-4a91-8b2f-ae5f73333113 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:50.364693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d5265-8f43-4e5f-9e95-5b76acdb4165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:50.365378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d5265-8f43-4e5f-9e95-5b76acdb4165 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:50.366187 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d5265-8f43-4e5f-9e95-5b76acdb4165 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:50.366465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d5265-8f43-4e5f-9e95-5b76acdb4165 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:50.367219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d5265-8f43-4e5f-9e95-5b76acdb4165 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:50.368629 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d5265-8f43-4e5f-9e95-5b76acdb4165 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:50.387743 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 498/1010] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jul 18 10:16:50 2022] POST /identity/v3/endpoints => generated 325 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:16:51.952190 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16de6fb-9179-481b-b7fd-8b89091a8f0c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:51.952810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16de6fb-9179-481b-b7fd-8b89091a8f0c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:51.952810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16de6fb-9179-481b-b7fd-8b89091a8f0c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:51.954196 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 513/1011] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:51.959429 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d354d3-f16e-428e-8f2e-e34a23bf6276 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:51.959833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d354d3-f16e-428e-8f2e-e34a23bf6276 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:51.960012 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d354d3-f16e-428e-8f2e-e34a23bf6276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:51.976432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-73d354d3-f16e-428e-8f2e-e34a23bf6276 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:51.977648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-73d354d3-f16e-428e-8f2e-e34a23bf6276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:52.057125 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 499/1012] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:51 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:52.079670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16de6fb-9179-481b-b7fd-8b89091a8f0c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:52.194437 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16de6fb-9179-481b-b7fd-8b89091a8f0c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:52.206895 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:52.233085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ba04f-34a0-4c84-9745-3f8489db4b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:52.233876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ba04f-34a0-4c84-9745-3f8489db4b84 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:52.234160 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ba04f-34a0-4c84-9745-3f8489db4b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:52.234435 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ba04f-34a0-4c84-9745-3f8489db4b84 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:52.235088 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ba04f-34a0-4c84-9745-3f8489db4b84 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:52.236454 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ba04f-34a0-4c84-9745-3f8489db4b84 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:52.248596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-609ba04f-34a0-4c84-9745-3f8489db4b84 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:16:52.248596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:52.248596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:52.248596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:52.248596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:52.248596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:52.248596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:52.248596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:52.248596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:16:52.248596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:52.248596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:52.248596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:52.248596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:52.248596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:52.248596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:52.248596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:52.248596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:52.249589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:52.249589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:52.249589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:52.249589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:52.249589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:52.249589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:52.249589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:52.249589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:52.249589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:52.249589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:52.249589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:52.249589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:52.249589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:52.249589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:52.249589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:52.249589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:52.250446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:52.250446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:52.250446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:52.250446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:52.250446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:52.250446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:52.250446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:52.250446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:52.250446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:52.250446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:52.250446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:52.250446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:52.250446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 10:16:52.250446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:52.252818 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 514/1013] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:52.255832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d354d3-f16e-428e-8f2e-e34a23bf6276 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:52.273260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:52.333460 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcc911-49c4-449b-8eae-ff206b038188 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:52.333792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcc911-49c4-449b-8eae-ff206b038188 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:52.334033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcc911-49c4-449b-8eae-ff206b038188 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:52.336996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 515/1014] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:52.341713 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f3824-f221-45c1-ac1f-ac9d3c52751b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:52.342199 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f3824-f221-45c1-ac1f-ac9d3c52751b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:52.342851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f3824-f221-45c1-ac1f-ac9d3c52751b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:52.363451 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d354d3-f16e-428e-8f2e-e34a23bf6276 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:52.369793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b17f3824-f221-45c1-ac1f-ac9d3c52751b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:52.369793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b17f3824-f221-45c1-ac1f-ac9d3c52751b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:52.405565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa62bb-e32f-4cf5-b810-06f1fdcfa859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:52.406232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa62bb-e32f-4cf5-b810-06f1fdcfa859 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:52.406517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa62bb-e32f-4cf5-b810-06f1fdcfa859 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:52.406793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa62bb-e32f-4cf5-b810-06f1fdcfa859 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:52.407498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa62bb-e32f-4cf5-b810-06f1fdcfa859 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:52.407806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa62bb-e32f-4cf5-b810-06f1fdcfa859 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:52.409212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa62bb-e32f-4cf5-b810-06f1fdcfa859 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:52.418819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 500/1015] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 18 10:16:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:52.419831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa62bb-e32f-4cf5-b810-06f1fdcfa859 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:52.445368 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 516/1016] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:52 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:52.447399 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c38b05-07a9-485f-a5bd-a24eebd205ca None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:52.447399 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c38b05-07a9-485f-a5bd-a24eebd205ca None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:52.447623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c38b05-07a9-485f-a5bd-a24eebd205ca None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:52.448754 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 517/1017] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:52.453974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774aa91a-139a-44fe-b479-6f52e68a70c3 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:52.454271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774aa91a-139a-44fe-b479-6f52e68a70c3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:52.454552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774aa91a-139a-44fe-b479-6f52e68a70c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:52.472273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-774aa91a-139a-44fe-b479-6f52e68a70c3 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:52.473463 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-774aa91a-139a-44fe-b479-6f52e68a70c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:52.503800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:52.526665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa62bb-e32f-4cf5-b810-06f1fdcfa859 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:52.542779 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 518/1018] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:52 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:16:52.543608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774aa91a-139a-44fe-b479-6f52e68a70c3 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:52.566244 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd12de3-d4f6-4bfb-890e-ca1a9f78bd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:52.567037 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd12de3-d4f6-4bfb-890e-ca1a9f78bd09 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:52.567331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd12de3-d4f6-4bfb-890e-ca1a9f78bd09 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:52.567614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd12de3-d4f6-4bfb-890e-ca1a9f78bd09 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:52.568335 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd12de3-d4f6-4bfb-890e-ca1a9f78bd09 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:52.568766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd12de3-d4f6-4bfb-890e-ca1a9f78bd09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:52.570165 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd12de3-d4f6-4bfb-890e-ca1a9f78bd09 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:52.580027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 501/1019] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jul 18 10:16:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:52.580182 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:52.580279 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd12de3-d4f6-4bfb-890e-ca1a9f78bd09 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:52.647128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774aa91a-139a-44fe-b479-6f52e68a70c3 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:52.674059 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd12de3-d4f6-4bfb-890e-ca1a9f78bd09 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:52.681318 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba9509-dd59-48fe-831d-41948e37d3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:52.682110 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba9509-dd59-48fe-831d-41948e37d3ed None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:52.682398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba9509-dd59-48fe-831d-41948e37d3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:52.682675 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba9509-dd59-48fe-831d-41948e37d3ed None admin] PATH_INFO: `/v3/services/image` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:52.683334 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba9509-dd59-48fe-831d-41948e37d3ed None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:52.684732 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba9509-dd59-48fe-831d-41948e37d3ed None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:52.697044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-edba9509-dd59-48fe-831d-41948e37d3ed None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:16:52.697044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:52.697044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:52.697044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:52.697044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:52.697044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:52.697044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:52.697044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:52.697044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:16:52.697044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:52.697044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:52.697044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:52.697044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:52.697044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:52.697044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:52.697044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:52.697044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:52.698125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:52.698125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:52.698125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:52.698125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:52.698125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:52.698125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:52.698125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:52.698125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:52.698125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:52.698125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:52.698125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:52.698125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:52.698125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:52.698125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:52.698125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:52.698125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:52.699069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:52.699069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:52.699069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:52.699069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:52.699069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:52.699069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:52.699069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:52.699069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:52.699069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:52.699069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:52.699069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:52.699069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:52.699069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:16:52.699069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:52.699069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 519/1020] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Jul 18 10:16:52 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:52.700084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba9509-dd59-48fe-831d-41948e37d3ed None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:52.711469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e371a-c99a-4bcf-9715-d4ba50be47b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:52.712253 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e371a-c99a-4bcf-9715-d4ba50be47b9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:52.712566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e371a-c99a-4bcf-9715-d4ba50be47b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:52.712842 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e371a-c99a-4bcf-9715-d4ba50be47b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:52.713998 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e371a-c99a-4bcf-9715-d4ba50be47b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:52.715419 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e371a-c99a-4bcf-9715-d4ba50be47b9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:52.721502 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 502/1021] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:52.721502 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e371a-c99a-4bcf-9715-d4ba50be47b9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:52.734797 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:52.800600 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:52.804733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba9509-dd59-48fe-831d-41948e37d3ed None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:52.826469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e371a-c99a-4bcf-9715-d4ba50be47b9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:52.837739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cf82c-8a3c-4135-9996-715c583a6ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:52.838387 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cf82c-8a3c-4135-9996-715c583a6ad1 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:52.838675 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cf82c-8a3c-4135-9996-715c583a6ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:52.838953 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376cf82c-8a3c-4135-9996-715c583a6ad1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:52.839690 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376cf82c-8a3c-4135-9996-715c583a6ad1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:52.841087 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376cf82c-8a3c-4135-9996-715c583a6ad1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:52.859930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 520/1022] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jul 18 10:16:52 2022] POST /identity/v3/endpoints => generated 334 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:16:52.860766 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cf82c-8a3c-4135-9996-715c583a6ad1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:52.874278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d49bfa-eb4e-4e68-b274-a978129650aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:52.875100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d49bfa-eb4e-4e68-b274-a978129650aa None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:52.875393 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d49bfa-eb4e-4e68-b274-a978129650aa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:52.875675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d49bfa-eb4e-4e68-b274-a978129650aa None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:52.876413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d49bfa-eb4e-4e68-b274-a978129650aa None admin] RBAC: Adding query filter params (name=image) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:52.876757 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d49bfa-eb4e-4e68-b274-a978129650aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:52.878197 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d49bfa-eb4e-4e68-b274-a978129650aa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:52.889679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 503/1023] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:52.890891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d49bfa-eb4e-4e68-b274-a978129650aa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:52.974097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:52.994324 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376cf82c-8a3c-4135-9996-715c583a6ad1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:53.009886 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d49bfa-eb4e-4e68-b274-a978129650aa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:53.035309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deaadff-d9bf-49d7-a9a9-e9acd31f26c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:53.036013 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deaadff-d9bf-49d7-a9a9-e9acd31f26c7 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:53.036304 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deaadff-d9bf-49d7-a9a9-e9acd31f26c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:53.036623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deaadff-d9bf-49d7-a9a9-e9acd31f26c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:53.037333 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deaadff-d9bf-49d7-a9a9-e9acd31f26c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:53.038970 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deaadff-d9bf-49d7-a9a9-e9acd31f26c7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:53.052290 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71452c6e-1d03-4e24-ba83-e2e03399d1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:53.053409 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71452c6e-1d03-4e24-ba83-e2e03399d1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:53.053693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71452c6e-1d03-4e24-ba83-e2e03399d1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:53.053972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71452c6e-1d03-4e24-ba83-e2e03399d1d7 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:53.054738 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71452c6e-1d03-4e24-ba83-e2e03399d1d7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:53.056047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71452c6e-1d03-4e24-ba83-e2e03399d1d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:53.065958 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71452c6e-1d03-4e24-ba83-e2e03399d1d7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:53.065958 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 504/1024] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:53.070521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71452c6e-1d03-4e24-ba83-e2e03399d1d7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:53.083650 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 521/1025] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 18 10:16:52 2022] POST /identity/v3/projects => generated 311 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:16:53.198491 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71452c6e-1d03-4e24-ba83-e2e03399d1d7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:53.237746 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501e972-473c-48fe-a5bf-0573b8bdd510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:53.238419 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501e972-473c-48fe-a5bf-0573b8bdd510 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:53.238706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501e972-473c-48fe-a5bf-0573b8bdd510 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:53.238988 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501e972-473c-48fe-a5bf-0573b8bdd510 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:53.239796 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501e972-473c-48fe-a5bf-0573b8bdd510 None admin] RBAC: Adding query filter params (interface=public, service_id=ca1d2ca51cc747188f6f2d54120939b3, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:53.240115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501e972-473c-48fe-a5bf-0573b8bdd510 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:53.241757 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501e972-473c-48fe-a5bf-0573b8bdd510 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:53.243659 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:53.248817 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 505/1026] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jul 18 10:16:53 2022] GET /identity/v3/endpoints?service_id=ca1d2ca51cc747188f6f2d54120939b3&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:53.254087 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deaadff-d9bf-49d7-a9a9-e9acd31f26c7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:53.265239 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:53.366744 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deaadff-d9bf-49d7-a9a9-e9acd31f26c7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:53.401467 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a5a29-1712-44ba-8330-43637cfd5a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:53.402131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a5a29-1712-44ba-8330-43637cfd5a2e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:53.402410 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a5a29-1712-44ba-8330-43637cfd5a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:53.402682 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a5a29-1712-44ba-8330-43637cfd5a2e None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:53.403365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a5a29-1712-44ba-8330-43637cfd5a2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:53.404757 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a5a29-1712-44ba-8330-43637cfd5a2e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:53.411396 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 522/1027] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jul 18 10:16:53 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:16:55.094725 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60369a81-07b2-482b-8d21-99ee36508071 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:55.095274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60369a81-07b2-482b-8d21-99ee36508071 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:55.095274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60369a81-07b2-482b-8d21-99ee36508071 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:55.096852 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 506/1028] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:55.102020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4492aa8-ded4-4495-bf9d-bb794194d31b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:55.102409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4492aa8-ded4-4495-bf9d-bb794194d31b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:55.102595 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4492aa8-ded4-4495-bf9d-bb794194d31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:55.120345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b4492aa8-ded4-4495-bf9d-bb794194d31b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:55.121568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b4492aa8-ded4-4495-bf9d-bb794194d31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:55.143622 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4739e5-f391-4caa-a745-1c856c19fa15 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:55.143912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4739e5-f391-4caa-a745-1c856c19fa15 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:55.144183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4739e5-f391-4caa-a745-1c856c19fa15 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:55.145641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 507/1029] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:55.151039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0c299-83e7-4f1e-8691-b7b8a1e9f5da None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:55.151332 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0c299-83e7-4f1e-8691-b7b8a1e9f5da None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:55.151615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0c299-83e7-4f1e-8691-b7b8a1e9f5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:55.170066 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2cf0c299-83e7-4f1e-8691-b7b8a1e9f5da None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:55.171256 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2cf0c299-83e7-4f1e-8691-b7b8a1e9f5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:55.201206 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:55.207098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 523/1030] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:55.230194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4492aa8-ded4-4495-bf9d-bb794194d31b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:55.243421 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 508/1031] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:16:55.268601 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0c299-83e7-4f1e-8691-b7b8a1e9f5da None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:55.340792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4492aa8-ded4-4495-bf9d-bb794194d31b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:55.374724 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0c299-83e7-4f1e-8691-b7b8a1e9f5da None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:55.378876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298be36f-77e7-4504-865d-a7c917fdf62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:55.379687 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298be36f-77e7-4504-865d-a7c917fdf62a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:55.379969 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298be36f-77e7-4504-865d-a7c917fdf62a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:55.380298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298be36f-77e7-4504-865d-a7c917fdf62a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:55.391763 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298be36f-77e7-4504-865d-a7c917fdf62a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:55.400292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-298be36f-77e7-4504-865d-a7c917fdf62a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:16:55.400292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:55.400292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:55.400292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:55.400292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:55.400292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:55.400292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:55.400292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:55.400292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:55.400292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:55.400292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:55.400292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:55.400292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:55.400292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:55.400292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:55.400292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:55.401452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:55.401452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:55.401452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:55.401452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:55.401452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:55.401452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:55.401452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:55.401452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:55.401452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:55.401452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:55.401452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:55.401452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:55.401452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:55.401452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:55.401452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:55.401452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:55.405405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:55.405405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:55.405405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:55.405405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:55.405405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:55.405405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:55.405405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:55.405405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:55.405405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:55.405405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:55.405405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:16:55.405405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:55.405405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 524/1032] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Jul 18 10:16:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:55.407414 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298be36f-77e7-4504-865d-a7c917fdf62a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:55.409273 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:55.418820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f448ff-1d19-4007-8c18-1ff2e9cd8b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:55.419616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f448ff-1d19-4007-8c18-1ff2e9cd8b74 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:55.419898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f448ff-1d19-4007-8c18-1ff2e9cd8b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:55.420171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f448ff-1d19-4007-8c18-1ff2e9cd8b74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:55.421376 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f448ff-1d19-4007-8c18-1ff2e9cd8b74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:55.422821 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f448ff-1d19-4007-8c18-1ff2e9cd8b74 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:55.424553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 509/1033] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:55.430598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f448ff-1d19-4007-8c18-1ff2e9cd8b74 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:55.470794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:55.512060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298be36f-77e7-4504-865d-a7c917fdf62a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:55.529864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f448ff-1d19-4007-8c18-1ff2e9cd8b74 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:55.545197 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7dd4c-ddae-4fc3-bb3a-b83c39cc59b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:55.545998 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7dd4c-ddae-4fc3-bb3a-b83c39cc59b9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:55.546282 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7dd4c-ddae-4fc3-bb3a-b83c39cc59b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:55.546551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7dd4c-ddae-4fc3-bb3a-b83c39cc59b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:55.547296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7dd4c-ddae-4fc3-bb3a-b83c39cc59b9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:55.547607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7dd4c-ddae-4fc3-bb3a-b83c39cc59b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:55.548999 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7dd4c-ddae-4fc3-bb3a-b83c39cc59b9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:55.556896 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 525/1034] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jul 18 10:16:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:55.558156 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c90730-9d22-4dc1-886b-6ba25496e429 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:55.558447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c90730-9d22-4dc1-886b-6ba25496e429 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:55.558727 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c90730-9d22-4dc1-886b-6ba25496e429 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:55.560121 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 526/1035] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:55.562566 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c90730-9d22-4dc1-886b-6ba25496e429 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:55.564904 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998b8a2-97e5-48c0-ae53-c05910a85cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:55.565606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998b8a2-97e5-48c0-ae53-c05910a85cda None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:55.565888 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998b8a2-97e5-48c0-ae53-c05910a85cda None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:55.566168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998b8a2-97e5-48c0-ae53-c05910a85cda None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:55.566882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998b8a2-97e5-48c0-ae53-c05910a85cda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:55.568305 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998b8a2-97e5-48c0-ae53-c05910a85cda None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:55.600606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 510/1036] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 18 10:16:55 2022] POST /identity/v3/users => generated 285 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:16:55.600606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22771f85-33af-49f4-ad8a-599ae918f75b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:55.600606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22771f85-33af-49f4-ad8a-599ae918f75b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:55.600606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22771f85-33af-49f4-ad8a-599ae918f75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:55.608959 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-22771f85-33af-49f4-ad8a-599ae918f75b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:55.610200 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-22771f85-33af-49f4-ad8a-599ae918f75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:55.663832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:55.679665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c90730-9d22-4dc1-886b-6ba25496e429 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:55.688731 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 511/1037] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:55.707070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:55.712519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22771f85-33af-49f4-ad8a-599ae918f75b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:55.722252 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c5c5c-f41a-43ea-953f-d2a91f7cb164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:55.723509 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c5c5c-f41a-43ea-953f-d2a91f7cb164 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:55.723799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c5c5c-f41a-43ea-953f-d2a91f7cb164 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:55.724087 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c5c5c-f41a-43ea-953f-d2a91f7cb164 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:55.736967 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c5c5c-f41a-43ea-953f-d2a91f7cb164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:55.739083 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c5c5c-f41a-43ea-953f-d2a91f7cb164 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:55.757523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a15c5c5c-f41a-43ea-953f-d2a91f7cb164 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:55.757523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:55.757523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:55.757523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:55.757523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:55.757523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:55.757523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:55.757523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:55.757523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:55.757523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:55.757523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:55.757523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:55.757523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:55.757523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:55.757523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:55.758471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:55.758471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:55.758471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:55.758471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:55.758471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:55.758471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:55.758471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:55.758471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:55.758471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:55.758471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:55.758471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:55.758471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:55.758471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:55.758471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:55.758471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:55.758471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:55.758471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:55.759389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:55.759389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:55.759389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:55.759389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:55.759389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 527/1038] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:55.764667 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c5c5c-f41a-43ea-953f-d2a91f7cb164 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:55.832040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22771f85-33af-49f4-ad8a-599ae918f75b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:55.868444 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8802dba-9f34-4dc5-9903-d561fa60430c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:55.869262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8802dba-9f34-4dc5-9903-d561fa60430c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:55.869544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8802dba-9f34-4dc5-9903-d561fa60430c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:55.869826 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8802dba-9f34-4dc5-9903-d561fa60430c None admin] PATH_INFO: `/v3/services/image` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:55.870477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8802dba-9f34-4dc5-9903-d561fa60430c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:55.871842 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8802dba-9f34-4dc5-9903-d561fa60430c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:55.882790 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c5c5c-f41a-43ea-953f-d2a91f7cb164 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:55.883833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a8802dba-9f34-4dc5-9903-d561fa60430c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:16:55.883833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:55.883833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:16:55.883833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:16:55.883833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:16:55.883833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:16:55.883833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:16:55.883833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:55.883833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:16:55.883833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:55.883833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:16:55.883833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:55.883833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:55.883833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:55.883833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:55.883833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:55.883833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:55.884858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:55.884858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:55.884858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:55.884858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:55.884858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:55.884858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:55.884858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:16:55.884858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:16:55.884858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:16:55.884858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:16:55.884858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:55.884858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:55.884858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:55.884858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:55.884858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:55.884858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:55.885753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:55.885753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:55.885753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:55.885753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:55.885753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:55.885753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:55.885753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:55.885753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:55.885753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:55.885753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:55.885753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:16:55.885753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:16:55.885753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 10:16:55.885753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:55.885753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 512/1039] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Jul 18 10:16:55 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:55.891085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8802dba-9f34-4dc5-9903-d561fa60430c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:55.922495 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9412e2-5b2a-4b37-abbe-40b8beec86d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:55.923072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9412e2-5b2a-4b37-abbe-40b8beec86d1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:55.923358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9412e2-5b2a-4b37-abbe-40b8beec86d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:55.923639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9412e2-5b2a-4b37-abbe-40b8beec86d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:55.924357 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9412e2-5b2a-4b37-abbe-40b8beec86d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:55.924695 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9412e2-5b2a-4b37-abbe-40b8beec86d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:55.926055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9412e2-5b2a-4b37-abbe-40b8beec86d1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:55.937735 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 528/1040] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:55.939824 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:55.943310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9412e2-5b2a-4b37-abbe-40b8beec86d1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:55.978354 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:55.999523 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8802dba-9f34-4dc5-9903-d561fa60430c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:56.039602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0bc30-2e39-4c7a-bc29-ff7a7fec0927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:56.040268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0bc30-2e39-4c7a-bc29-ff7a7fec0927 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:56.040579 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0bc30-2e39-4c7a-bc29-ff7a7fec0927 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:56.040866 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0bc30-2e39-4c7a-bc29-ff7a7fec0927 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:56.041573 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0bc30-2e39-4c7a-bc29-ff7a7fec0927 None admin] RBAC: Adding query filter params (name=image) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:56.041930 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0bc30-2e39-4c7a-bc29-ff7a7fec0927 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:56.043308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0bc30-2e39-4c7a-bc29-ff7a7fec0927 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:56.048830 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 513/1041] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:56.050564 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9412e2-5b2a-4b37-abbe-40b8beec86d1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:56.054432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0bc30-2e39-4c7a-bc29-ff7a7fec0927 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:56.089021 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023c143-5949-4196-adf7-81e003f031da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:56.089816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023c143-5949-4196-adf7-81e003f031da None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:56.090103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023c143-5949-4196-adf7-81e003f031da None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:56.090381 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023c143-5949-4196-adf7-81e003f031da None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:56.098231 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023c143-5949-4196-adf7-81e003f031da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:56.099673 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023c143-5949-4196-adf7-81e003f031da None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:56.114204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-5023c143-5949-4196-adf7-81e003f031da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:16:56.114204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:56.114204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:56.114204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:56.114204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:56.114204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:56.114204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:56.114204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:56.114204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:56.114204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:56.114204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:56.114204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:56.114204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:16:56.114204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:16:56.114204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:16:56.114204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:16:56.116119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:56.116119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:56.116119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:16:56.116119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:56.116119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:16:56.116119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:16:56.116119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:56.116119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:56.116119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:56.116119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:56.116119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:56.116119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:56.116119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:56.116119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:56.116119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:56.116119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:56.116119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:56.116119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:56.117327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:56.117327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:56.117327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:16:56.117327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:16:56.117327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:16:56.117327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:16:56.117327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:16:56.117327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:16:56.117327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:16:56.117327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:56.117327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 529/1042] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jul 18 10:16:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:56.121093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023c143-5949-4196-adf7-81e003f031da None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:56.165890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0bc30-2e39-4c7a-bc29-ff7a7fec0927 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:56.182134 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:56.202268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a27f6-1357-41be-8f54-264354cd8b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:56.203062 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a27f6-1357-41be-8f54-264354cd8b40 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:56.203242 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a27f6-1357-41be-8f54-264354cd8b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:56.203530 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a27f6-1357-41be-8f54-264354cd8b40 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:56.204245 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a27f6-1357-41be-8f54-264354cd8b40 None admin] RBAC: Adding query filter params (type=image) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:56.204581 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a27f6-1357-41be-8f54-264354cd8b40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:56.205956 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a27f6-1357-41be-8f54-264354cd8b40 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:56.211680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 514/1043] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:56.217203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a27f6-1357-41be-8f54-264354cd8b40 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:56.234210 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:56.241856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023c143-5949-4196-adf7-81e003f031da None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:56.279117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fe91b-f6ec-4584-8b6d-530f228ac050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:56.280053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fe91b-f6ec-4584-8b6d-530f228ac050 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:56.280518 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fe91b-f6ec-4584-8b6d-530f228ac050 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:56.280813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fe91b-f6ec-4584-8b6d-530f228ac050 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:56.281662 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fe91b-f6ec-4584-8b6d-530f228ac050 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:56.281978 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fe91b-f6ec-4584-8b6d-530f228ac050 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:56.283415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fe91b-f6ec-4584-8b6d-530f228ac050 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:56.301545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 530/1044] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jul 18 10:16:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:56.307642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fe91b-f6ec-4584-8b6d-530f228ac050 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:56.326268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a27f6-1357-41be-8f54-264354cd8b40 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:56.361756 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c1e7e-1551-484d-a12a-856ff1253dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:56.362510 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c1e7e-1551-484d-a12a-856ff1253dcc None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:56.362796 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c1e7e-1551-484d-a12a-856ff1253dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:56.363103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c1e7e-1551-484d-a12a-856ff1253dcc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:56.363845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c1e7e-1551-484d-a12a-856ff1253dcc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:56.365239 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c1e7e-1551-484d-a12a-856ff1253dcc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:56.396537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 515/1045] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jul 18 10:16:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:16:56.427082 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fe91b-f6ec-4584-8b6d-530f228ac050 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:56.472935 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01428643-d8e8-4e6c-b0d7-da9b07593b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:56.473520 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01428643-d8e8-4e6c-b0d7-da9b07593b23 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:56.473809 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01428643-d8e8-4e6c-b0d7-da9b07593b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:56.474085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01428643-d8e8-4e6c-b0d7-da9b07593b23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:56.495029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:56.551877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01428643-d8e8-4e6c-b0d7-da9b07593b23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:56.553428 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01428643-d8e8-4e6c-b0d7-da9b07593b23 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:56.652988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 531/1046] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:16:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 10:16:56.659030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c1e7e-1551-484d-a12a-856ff1253dcc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:56.690624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:56.764614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c1e7e-1551-484d-a12a-856ff1253dcc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:56.801137 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f7734-7ecf-4752-8a88-16539eb74b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:56.801931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f7734-7ecf-4752-8a88-16539eb74b31 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:56.802210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f7734-7ecf-4752-8a88-16539eb74b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:56.802480 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f7734-7ecf-4752-8a88-16539eb74b31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:56.814304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f7734-7ecf-4752-8a88-16539eb74b31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:56.815753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f7734-7ecf-4752-8a88-16539eb74b31 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:56.833017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-629f7734-7ecf-4752-8a88-16539eb74b31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:56.833017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:56.833017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:56.833017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:56.833017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:56.833017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:56.833017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:56.833017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:56.833017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:56.833017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:56.833017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:56.833017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:56.833017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:56.833017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:56.833017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:56.833933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:56.833933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:56.833933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:56.833933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:56.833933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:56.833933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:56.833933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:56.833933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:56.833933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:56.833933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:56.833933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:56.833933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:56.833933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:56.833933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:56.833933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:56.833933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:56.833933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:56.834905 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:56.834905 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:56.834905 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:56.834905 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:56.835691 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 516/1047] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:56.840039 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01428643-d8e8-4e6c-b0d7-da9b07593b23 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:56.891599 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:56.943519 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01428643-d8e8-4e6c-b0d7-da9b07593b23 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:56.979379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ea229-55bc-4219-8860-c4f8f4e8835e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:56.980026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ea229-55bc-4219-8860-c4f8f4e8835e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:56.980308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ea229-55bc-4219-8860-c4f8f4e8835e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:56.981023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ea229-55bc-4219-8860-c4f8f4e8835e None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:56.981585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ea229-55bc-4219-8860-c4f8f4e8835e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:56.981945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ea229-55bc-4219-8860-c4f8f4e8835e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:56.983353 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ea229-55bc-4219-8860-c4f8f4e8835e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:56.996585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 532/1048] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:57.001165 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f7734-7ecf-4752-8a88-16539eb74b31 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:57.088024 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:57.109402 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f7734-7ecf-4752-8a88-16539eb74b31 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:57.145416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37be56-6454-4601-aacf-d636bb4ee6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:57.146207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37be56-6454-4601-aacf-d636bb4ee6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:57.146488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37be56-6454-4601-aacf-d636bb4ee6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:57.146957 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37be56-6454-4601-aacf-d636bb4ee6a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:57.154377 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37be56-6454-4601-aacf-d636bb4ee6a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:57.156003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37be56-6454-4601-aacf-d636bb4ee6a0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:57.171204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-de37be56-6454-4601-aacf-d636bb4ee6a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:57.171204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:57.171204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:57.171204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:57.171204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:57.171204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:57.171204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:57.171204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:57.171204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:57.171204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:57.171204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:57.171204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:57.171204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:16:57.171204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:16:57.171204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:16:57.180968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:16:57.180968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:57.180968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:57.180968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:57.180968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:57.180968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:57.180968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:57.180968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:57.180968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:57.180968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:57.180968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:57.180968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:57.180968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:57.180968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:57.180968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:57.180968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:57.180968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:57.181926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:16:57.181926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:16:57.181926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:16:57.181926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:16:57.181926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:16:57.181926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:16:57.181926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:16:57.181926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:16:57.181926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 517/1049] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:16:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:16:57.182433 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ea229-55bc-4219-8860-c4f8f4e8835e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:57.284892 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ea229-55bc-4219-8860-c4f8f4e8835e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:57.293911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:57.321461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e6e07-7ad9-4bd1-8ef0-93ff75554d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:57.322120 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e6e07-7ad9-4bd1-8ef0-93ff75554d85 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:57.322401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e6e07-7ad9-4bd1-8ef0-93ff75554d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:57.322674 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e6e07-7ad9-4bd1-8ef0-93ff75554d85 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:57.323464 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e6e07-7ad9-4bd1-8ef0-93ff75554d85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:57.323772 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e6e07-7ad9-4bd1-8ef0-93ff75554d85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:57.325225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e6e07-7ad9-4bd1-8ef0-93ff75554d85 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:57.342203 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 533/1050] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:16:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:57.343262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37be56-6454-4601-aacf-d636bb4ee6a0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:57.443427 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37be56-6454-4601-aacf-d636bb4ee6a0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:57.477065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a94bde-9857-4c44-9bf1-10f21bdb00a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:57.477708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a94bde-9857-4c44-9bf1-10f21bdb00a6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:57.477989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a94bde-9857-4c44-9bf1-10f21bdb00a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:57.478262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a94bde-9857-4c44-9bf1-10f21bdb00a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:57.479085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a94bde-9857-4c44-9bf1-10f21bdb00a6 None admin] RBAC: Adding query filter params (role.id=25bb48ece7784b5d9b0a60db78e81edd, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=29048c2021e340b081ae44a390583e7a) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:57.479395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a94bde-9857-4c44-9bf1-10f21bdb00a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:57.481099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a94bde-9857-4c44-9bf1-10f21bdb00a6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:57.492756 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 518/1051] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:16:57 2022] GET /identity/v3/role_assignments?user.id=29048c2021e340b081ae44a390583e7a&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=25bb48ece7784b5d9b0a60db78e81edd => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:57.695495 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4c89e-8c1c-45e6-9ebe-dab4a2e02a35 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:57.695891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4c89e-8c1c-45e6-9ebe-dab4a2e02a35 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:57.696077 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4c89e-8c1c-45e6-9ebe-dab4a2e02a35 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:57.697623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 534/1052] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:57.702663 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527fda8e-53d0-4563-8c9b-92a7ebd03662 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:57.702962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527fda8e-53d0-4563-8c9b-92a7ebd03662 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:57.703239 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527fda8e-53d0-4563-8c9b-92a7ebd03662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:57.715787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:57.722376 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-527fda8e-53d0-4563-8c9b-92a7ebd03662 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:57.723970 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-527fda8e-53d0-4563-8c9b-92a7ebd03662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:57.791172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 519/1053] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:57.815388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4c89e-8c1c-45e6-9ebe-dab4a2e02a35 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:57.916397 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4c89e-8c1c-45e6-9ebe-dab4a2e02a35 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:57.952625 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd7313-d5b3-407a-8683-8cd983ac6bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:57.953336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd7313-d5b3-407a-8683-8cd983ac6bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:57.953618 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd7313-d5b3-407a-8683-8cd983ac6bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:57.953890 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd7313-d5b3-407a-8683-8cd983ac6bc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:57.964331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd7313-d5b3-407a-8683-8cd983ac6bc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:57.971871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-16cd7313-d5b3-407a-8683-8cd983ac6bc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:16:57.971871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:57.971871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:57.971871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:57.971871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:57.971871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:57.971871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:57.971871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:57.971871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:57.971871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:57.971871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:57.971871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:57.971871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:57.971871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:57.971871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:57.971871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:57.973026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:57.973026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:57.973026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:57.973026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:57.973026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:57.973026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:57.973026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:57.973026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:57.973026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:57.973026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:57.973026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:57.973026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:57.973026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:57.973026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:57.973026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:57.973026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:57.973929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:57.973929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:57.973929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:57.973929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:57.973929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:57.973929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:57.973929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:57.973929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:57.973929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:57.973929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:57.973929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:16:57.973929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:57.973929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 535/1054] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:16:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:57.981523 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527fda8e-53d0-4563-8c9b-92a7ebd03662 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:58.078955 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527fda8e-53d0-4563-8c9b-92a7ebd03662 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:58.112737 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214f375-e8de-4fe7-9456-4f5dd1724891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:58.113409 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214f375-e8de-4fe7-9456-4f5dd1724891 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:58.113698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214f375-e8de-4fe7-9456-4f5dd1724891 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:58.113975 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214f375-e8de-4fe7-9456-4f5dd1724891 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:58.114724 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5214f375-e8de-4fe7-9456-4f5dd1724891 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:58.115034 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5214f375-e8de-4fe7-9456-4f5dd1724891 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:58.116978 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5214f375-e8de-4fe7-9456-4f5dd1724891 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:58.119239 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:58.132555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 520/1055] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:16:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:58.133200 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd7313-d5b3-407a-8683-8cd983ac6bc7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:58.167498 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:58.247891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd7313-d5b3-407a-8683-8cd983ac6bc7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:58.281622 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744075fa-e10a-459f-91ea-d0766845144b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:58.282501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744075fa-e10a-459f-91ea-d0766845144b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:58.282788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744075fa-e10a-459f-91ea-d0766845144b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:58.283111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744075fa-e10a-459f-91ea-d0766845144b None admin] PATH_INFO: `/v3/users/7833119558c44d8bae729d0ad6a2f39b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:58.297254 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744075fa-e10a-459f-91ea-d0766845144b None admin] RBAC: Authorizing `identity:get_user(user_id=7833119558c44d8bae729d0ad6a2f39b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:58.298699 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744075fa-e10a-459f-91ea-d0766845144b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:58.305611 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 536/1056] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:16:58 2022] GET /identity/v3/users/7833119558c44d8bae729d0ad6a2f39b => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:58.307121 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214f375-e8de-4fe7-9456-4f5dd1724891 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:58.407025 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5214f375-e8de-4fe7-9456-4f5dd1724891 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:58.440118 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b894a3f-d02f-411e-b223-9d5147fc377a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:58.440785 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b894a3f-d02f-411e-b223-9d5147fc377a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:58.441072 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b894a3f-d02f-411e-b223-9d5147fc377a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:58.441347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b894a3f-d02f-411e-b223-9d5147fc377a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:58.467599 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f69b2e-9847-4dcc-a601-fca28cc699ee None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:58.467887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f69b2e-9847-4dcc-a601-fca28cc699ee None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:58.468165 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f69b2e-9847-4dcc-a601-fca28cc699ee None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:58.469588 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 537/1057] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:58.474297 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72e62c-a94b-427c-bc64-3aa5e2eeb1aa None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:58.474590 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72e62c-a94b-427c-bc64-3aa5e2eeb1aa None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:58.474900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72e62c-a94b-427c-bc64-3aa5e2eeb1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:58.491933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9d72e62c-a94b-427c-bc64-3aa5e2eeb1aa None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:58.493166 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9d72e62c-a94b-427c-bc64-3aa5e2eeb1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:58.508811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:58.513812 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b894a3f-d02f-411e-b223-9d5147fc377a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:58.515693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b894a3f-d02f-411e-b223-9d5147fc377a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:58.566343 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 538/1058] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:16:58.589677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72e62c-a94b-427c-bc64-3aa5e2eeb1aa None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:58.607815 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:58.630971 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 521/1059] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:16:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:16:58.636918 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b894a3f-d02f-411e-b223-9d5147fc377a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:58.694613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72e62c-a94b-427c-bc64-3aa5e2eeb1aa None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:58.731268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b29a4-345a-4f19-992a-2d6522f8c1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:58.731915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b29a4-345a-4f19-992a-2d6522f8c1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:58.732236 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b29a4-345a-4f19-992a-2d6522f8c1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:58.732536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b29a4-345a-4f19-992a-2d6522f8c1e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:58.739606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:58.744666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b894a3f-d02f-411e-b223-9d5147fc377a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:58.781728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41559d-9093-4f7c-a51c-c28b3cfe628f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:58.782506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41559d-9093-4f7c-a51c-c28b3cfe628f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:58.782800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41559d-9093-4f7c-a51c-c28b3cfe628f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:58.783065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41559d-9093-4f7c-a51c-c28b3cfe628f None admin] PATH_INFO: `/v3/projects/438aa38c0c534fb9bbd18d9d4d4c3526` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:58.784523 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41559d-9093-4f7c-a51c-c28b3cfe628f None admin] RBAC: Authorizing `identity:get_project(project_id=438aa38c0c534fb9bbd18d9d4d4c3526)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:58.785951 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41559d-9093-4f7c-a51c-c28b3cfe628f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:58.788363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 522/1060] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:16:58 2022] GET /identity/v3/projects/438aa38c0c534fb9bbd18d9d4d4c3526 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:58.794494 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41559d-9093-4f7c-a51c-c28b3cfe628f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:58.807273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b29a4-345a-4f19-992a-2d6522f8c1e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:58.808753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b29a4-345a-4f19-992a-2d6522f8c1e4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:58.826736 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:58.899373 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41559d-9093-4f7c-a51c-c28b3cfe628f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:58.922945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 539/1061] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:16:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 10:16:58.924131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b29a4-345a-4f19-992a-2d6522f8c1e4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:58.939029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db83867-e3d2-4698-a561-47ac68f3771c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:58.939683 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db83867-e3d2-4698-a561-47ac68f3771c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:58.939964 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db83867-e3d2-4698-a561-47ac68f3771c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:58.940236 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db83867-e3d2-4698-a561-47ac68f3771c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:58.941095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db83867-e3d2-4698-a561-47ac68f3771c None admin] RBAC: Adding query filter params (role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.project.id=438aa38c0c534fb9bbd18d9d4d4c3526, user.id=7833119558c44d8bae729d0ad6a2f39b) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:58.941410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db83867-e3d2-4698-a561-47ac68f3771c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:58.942828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db83867-e3d2-4698-a561-47ac68f3771c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:58.956548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 523/1062] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:16:58 2022] GET /identity/v3/role_assignments?user.id=7833119558c44d8bae729d0ad6a2f39b&scope.project.id=438aa38c0c534fb9bbd18d9d4d4c3526&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:16:59.049852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b29a4-345a-4f19-992a-2d6522f8c1e4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:59.068068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:59.089407 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ff027-4cfb-4570-a103-418f236c81ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:59.090307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ff027-4cfb-4570-a103-418f236c81ed None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:59.090652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ff027-4cfb-4570-a103-418f236c81ed None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:59.091060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ff027-4cfb-4570-a103-418f236c81ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:59.102917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ff027-4cfb-4570-a103-418f236c81ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:59.104549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ff027-4cfb-4570-a103-418f236c81ed None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:59.123597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d72ff027-4cfb-4570-a103-418f236c81ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:59.123597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:59.123597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:59.123597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:59.123597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:59.123597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:59.123597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:59.123597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:59.123597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:59.123597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:59.123597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:59.123597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:59.123597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:16:59.123597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:16:59.123597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:16:59.124648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:16:59.124648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:59.124648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:59.124648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:59.124648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:59.124648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:59.124648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:59.124648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:59.124648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:59.124648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:59.124648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:59.124648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:59.124648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:59.124648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:59.124648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:59.124648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:59.124648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:59.125626 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:16:59.125626 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:16:59.125626 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:16:59.125626 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:59.125626 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 540/1063] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:59.132528 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db83867-e3d2-4698-a561-47ac68f3771c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:59.164842 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:59.239768 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db83867-e3d2-4698-a561-47ac68f3771c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:59.277836 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019b981-7ae6-47bf-8907-9b1c03837b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:59.278498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019b981-7ae6-47bf-8907-9b1c03837b0c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:59.278782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019b981-7ae6-47bf-8907-9b1c03837b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:59.279056 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019b981-7ae6-47bf-8907-9b1c03837b0c None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:59.279774 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019b981-7ae6-47bf-8907-9b1c03837b0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:59.280080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019b981-7ae6-47bf-8907-9b1c03837b0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:59.281471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019b981-7ae6-47bf-8907-9b1c03837b0c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:59.300623 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 524/1064] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:59.499299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4139d-599c-4e5f-8cd0-e399286d03ef None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:59.499823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4139d-599c-4e5f-8cd0-e399286d03ef None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:59.499823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4139d-599c-4e5f-8cd0-e399286d03ef None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:59.501238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 541/1065] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:16:59.506997 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f51eb-3fc2-44bf-8d9c-05959deb1c5b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:59.507415 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f51eb-3fc2-44bf-8d9c-05959deb1c5b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:59.507592 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f51eb-3fc2-44bf-8d9c-05959deb1c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:59.524206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-193f51eb-3fc2-44bf-8d9c-05959deb1c5b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:16:59.525431 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-193f51eb-3fc2-44bf-8d9c-05959deb1c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:16:59.592790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 525/1066] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:16:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:16:59.615133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4139d-599c-4e5f-8cd0-e399286d03ef None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:59.715377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4139d-599c-4e5f-8cd0-e399286d03ef None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:59.749151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b9392-3bfc-4c53-a2f0-15f7e6cefd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:59.749953 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b9392-3bfc-4c53-a2f0-15f7e6cefd4e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:59.750240 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b9392-3bfc-4c53-a2f0-15f7e6cefd4e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:59.750519 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b9392-3bfc-4c53-a2f0-15f7e6cefd4e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:59.760553 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b9392-3bfc-4c53-a2f0-15f7e6cefd4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:16:59.768789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-195b9392-3bfc-4c53-a2f0-15f7e6cefd4e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:16:59.768789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:16:59.768789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:16:59.768789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:16:59.768789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:16:59.768789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:16:59.768789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:16:59.768789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:16:59.768789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:16:59.768789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:16:59.768789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:16:59.768789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:16:59.768789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:16:59.768789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:16:59.768789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:16:59.768789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:16:59.769852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:16:59.769852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:16:59.769852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:16:59.769852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:16:59.769852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:16:59.769852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:16:59.769852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:16:59.769852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:16:59.769852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:16:59.769852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:16:59.769852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:16:59.769852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:16:59.769852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:16:59.769852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:16:59.769852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:16:59.769852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:16:59.770716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:16:59.770716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:16:59.770716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:16:59.770716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:16:59.770716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:16:59.770716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:16:59.770716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:16:59.770716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:16:59.770716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:16:59.770716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:16:59.770716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:16:59.770716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:16:59.770716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 542/1067] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Jul 18 10:16:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:16:59.776869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f51eb-3fc2-44bf-8d9c-05959deb1c5b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:59.802176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:16:59.883590 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f51eb-3fc2-44bf-8d9c-05959deb1c5b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:16:59.916218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a1524-9deb-4cc1-a2a9-6e5d47f3d9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:16:59.916899 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a1524-9deb-4cc1-a2a9-6e5d47f3d9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:16:59.917183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a1524-9deb-4cc1-a2a9-6e5d47f3d9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:16:59.917458 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a1524-9deb-4cc1-a2a9-6e5d47f3d9a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:16:59.918196 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a1524-9deb-4cc1-a2a9-6e5d47f3d9a9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:16:59.918502 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a1524-9deb-4cc1-a2a9-6e5d47f3d9a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:16:59.919865 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a1524-9deb-4cc1-a2a9-6e5d47f3d9a9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:16:59.927501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 526/1068] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jul 18 10:16:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:16:59.933602 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b9392-3bfc-4c53-a2f0-15f7e6cefd4e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:16:59.962206 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:00.033034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b9392-3bfc-4c53-a2f0-15f7e6cefd4e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:00.069477 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6aa4bf-c066-4c2b-8741-5ccea03d49e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:00.070546 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6aa4bf-c066-4c2b-8741-5ccea03d49e6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:00.071051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6aa4bf-c066-4c2b-8741-5ccea03d49e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:00.071548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6aa4bf-c066-4c2b-8741-5ccea03d49e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:00.083935 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6aa4bf-c066-4c2b-8741-5ccea03d49e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:00.085436 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6aa4bf-c066-4c2b-8741-5ccea03d49e6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:00.101996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-1c6aa4bf-c066-4c2b-8741-5ccea03d49e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:17:00.101996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:00.101996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:17:00.101996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:17:00.101996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:17:00.101996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:17:00.101996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:17:00.101996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:17:00.101996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:17:00.101996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:17:00.101996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:17:00.101996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:17:00.101996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:17:00.101996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:17:00.101996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:17:00.102913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:17:00.102913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:17:00.102913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:17:00.102913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:17:00.102913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:17:00.102913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:17:00.102913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:17:00.102913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:17:00.102913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:17:00.102913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:17:00.102913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:17:00.102913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:17:00.102913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:17:00.102913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:17:00.102913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:17:00.102913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:17:00.102913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:17:00.103819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:17:00.103819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:17:00.103819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:17:00.103819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:17:00.104661 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 543/1069] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:16:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:17:00.109082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a1524-9deb-4cc1-a2a9-6e5d47f3d9a9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:00.192408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:00.210733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a1524-9deb-4cc1-a2a9-6e5d47f3d9a9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:00.243052 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ed32d-eec2-47f1-b5f6-d1ee5393908d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:00.243702 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ed32d-eec2-47f1-b5f6-d1ee5393908d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:00.243985 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ed32d-eec2-47f1-b5f6-d1ee5393908d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:00.244260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ed32d-eec2-47f1-b5f6-d1ee5393908d None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:00.245008 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ed32d-eec2-47f1-b5f6-d1ee5393908d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:00.245316 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ed32d-eec2-47f1-b5f6-d1ee5393908d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:00.246667 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ed32d-eec2-47f1-b5f6-d1ee5393908d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:00.258641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 527/1070] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:00.264152 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6aa4bf-c066-4c2b-8741-5ccea03d49e6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:00.342753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:00.362109 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6aa4bf-c066-4c2b-8741-5ccea03d49e6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:00.394829 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3dec2-11a7-4b65-b115-d30c156e75b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:00.395609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3dec2-11a7-4b65-b115-d30c156e75b1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:00.395891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3dec2-11a7-4b65-b115-d30c156e75b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:00.396161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3dec2-11a7-4b65-b115-d30c156e75b1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:00.404102 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3dec2-11a7-4b65-b115-d30c156e75b1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:00.405608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3dec2-11a7-4b65-b115-d30c156e75b1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:00.419026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-21d3dec2-11a7-4b65-b115-d30c156e75b1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:17:00.419026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:00.419026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:17:00.419026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:17:00.419026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:17:00.419026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:17:00.419026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:17:00.419026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:17:00.419026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:17:00.419026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:17:00.419026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:17:00.419026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:17:00.419026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:17:00.419026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:17:00.419026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:17:00.419026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:17:00.420159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:17:00.420159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:17:00.420159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:17:00.420159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:17:00.420159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:17:00.420159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:17:00.420159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:17:00.420159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:17:00.420159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:17:00.420159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:17:00.420159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:17:00.420159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:17:00.420159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:17:00.420159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:17:00.420159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:17:00.420159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:17:00.420159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:17:00.420159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:17:00.421215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:17:00.421215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:17:00.421215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:17:00.421215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:17:00.421215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:17:00.421215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:17:00.421215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:17:00.421215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:17:00.421215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:17:00.421215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:17:00.421215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 544/1071] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jul 18 10:17:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:17:00.426160 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ed32d-eec2-47f1-b5f6-d1ee5393908d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:00.526432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ed32d-eec2-47f1-b5f6-d1ee5393908d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:00.558427 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f28b6f-a3f2-4848-9b61-d02ee2c4c641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:00.559093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f28b6f-a3f2-4848-9b61-d02ee2c4c641 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:00.559376 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f28b6f-a3f2-4848-9b61-d02ee2c4c641 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:00.559651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f28b6f-a3f2-4848-9b61-d02ee2c4c641 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:00.560477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f28b6f-a3f2-4848-9b61-d02ee2c4c641 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:00.560818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f28b6f-a3f2-4848-9b61-d02ee2c4c641 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:00.562232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f28b6f-a3f2-4848-9b61-d02ee2c4c641 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:00.576298 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:00.579856 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 528/1072] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jul 18 10:17:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:00.586242 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3dec2-11a7-4b65-b115-d30c156e75b1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:00.683343 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3dec2-11a7-4b65-b115-d30c156e75b1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:00.716300 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca346e-f300-4064-be8e-17b0844513bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:00.717120 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca346e-f300-4064-be8e-17b0844513bc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:00.717408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca346e-f300-4064-be8e-17b0844513bc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:00.717686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca346e-f300-4064-be8e-17b0844513bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:00.726182 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:00.728499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca346e-f300-4064-be8e-17b0844513bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:00.729952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca346e-f300-4064-be8e-17b0844513bc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:00.747220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d9ca346e-f300-4064-be8e-17b0844513bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:17:00.747220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:00.747220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:17:00.747220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:17:00.747220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:17:00.747220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:17:00.747220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:17:00.747220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:17:00.747220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:17:00.747220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:17:00.747220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:17:00.747220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:17:00.747220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:17:00.747220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:17:00.747220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:17:00.748183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:17:00.748183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:17:00.748183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:17:00.748183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:17:00.748183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:17:00.748183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:17:00.748183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:17:00.748183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:17:00.748183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:17:00.748183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:17:00.748183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:17:00.748183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:17:00.748183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:17:00.748183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:17:00.748183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:17:00.748183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:17:00.748183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:17:00.749108 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:17:00.749108 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:17:00.749108 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:17:00.749108 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:17:00.749108 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 545/1073] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:17:00.754277 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f28b6f-a3f2-4848-9b61-d02ee2c4c641 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:00.852798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f28b6f-a3f2-4848-9b61-d02ee2c4c641 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:00.886504 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d1122-a289-4163-baa8-223d96ae0658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:00.887153 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d1122-a289-4163-baa8-223d96ae0658 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:00.887434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d1122-a289-4163-baa8-223d96ae0658 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:00.887709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d1122-a289-4163-baa8-223d96ae0658 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:00.888424 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d1122-a289-4163-baa8-223d96ae0658 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:00.888757 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d1122-a289-4163-baa8-223d96ae0658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:00.890120 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d1122-a289-4163-baa8-223d96ae0658 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:00.901318 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 529/1074] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:00.907242 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca346e-f300-4064-be8e-17b0844513bc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:00.978804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadddd9-d294-4865-b2c2-bc9f2c1c3284 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:00.979131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadddd9-d294-4865-b2c2-bc9f2c1c3284 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:00.979415 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadddd9-d294-4865-b2c2-bc9f2c1c3284 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:00.981499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 530/1075] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:17:00.988010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7c74b-4913-418e-98da-79ae7a3c8c27 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:00.988299 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7c74b-4913-418e-98da-79ae7a3c8c27 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:00.988606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7c74b-4913-418e-98da-79ae7a3c8c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:01.007209 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-09f7c74b-4913-418e-98da-79ae7a3c8c27 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:01.008418 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-09f7c74b-4913-418e-98da-79ae7a3c8c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:01.008921 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca346e-f300-4064-be8e-17b0844513bc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:01.041195 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac62f13-eb97-4728-a9b5-9e0f5cdca433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:01.041975 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac62f13-eb97-4728-a9b5-9e0f5cdca433 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:01.042258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac62f13-eb97-4728-a9b5-9e0f5cdca433 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:01.042534 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac62f13-eb97-4728-a9b5-9e0f5cdca433 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:01.049069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac62f13-eb97-4728-a9b5-9e0f5cdca433 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:01.050525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac62f13-eb97-4728-a9b5-9e0f5cdca433 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:01.063510 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:01.065122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-eac62f13-eb97-4728-a9b5-9e0f5cdca433 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:17:01.065122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:01.065122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:17:01.065122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:17:01.065122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:17:01.065122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:17:01.065122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:17:01.065122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:17:01.065122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:17:01.065122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:17:01.065122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:17:01.065122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:17:01.065122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:17:01.065122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:17:01.065122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:17:01.066268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:17:01.066268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:17:01.066268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:17:01.066268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:17:01.066268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:17:01.066268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:17:01.066268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:17:01.066268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:17:01.066268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:17:01.066268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:17:01.066268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:17:01.066268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:17:01.066268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:17:01.066268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:17:01.066268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:17:01.066268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:17:01.066268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:17:01.067551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:17:01.067551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:17:01.067551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:17:01.067551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:17:01.067551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:17:01.067551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:17:01.067551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:17:01.067551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:17:01.067551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 546/1076] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:17:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:17:01.072636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac62f13-eb97-4728-a9b5-9e0f5cdca433 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:01.093348 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 531/1077] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:17:01.101581 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7c74b-4913-418e-98da-79ae7a3c8c27 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:01.136006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:01.184721 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac62f13-eb97-4728-a9b5-9e0f5cdca433 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:01.201378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7c74b-4913-418e-98da-79ae7a3c8c27 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:01.218539 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a88664-443a-4f32-a8f7-8d0cd6f9582e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:01.219194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a88664-443a-4f32-a8f7-8d0cd6f9582e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:01.219480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a88664-443a-4f32-a8f7-8d0cd6f9582e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:01.219761 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a88664-443a-4f32-a8f7-8d0cd6f9582e None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:01.220896 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a88664-443a-4f32-a8f7-8d0cd6f9582e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:01.220896 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a88664-443a-4f32-a8f7-8d0cd6f9582e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:01.222310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a88664-443a-4f32-a8f7-8d0cd6f9582e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:01.234475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd21bd-8365-4dd3-be18-e0743260e634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:01.235022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 547/1078] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:17:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:17:01.235210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd21bd-8365-4dd3-be18-e0743260e634 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:01.235492 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd21bd-8365-4dd3-be18-e0743260e634 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:01.235781 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd21bd-8365-4dd3-be18-e0743260e634 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:01.241232 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a88664-443a-4f32-a8f7-8d0cd6f9582e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:01.246509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd21bd-8365-4dd3-be18-e0743260e634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:17:01.254094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0acd21bd-8365-4dd3-be18-e0743260e634 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:17:01.254094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:01.254094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:17:01.254094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:17:01.254094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:17:01.254094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:17:01.254094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:17:01.254094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:17:01.254094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:17:01.254094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:17:01.254094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:17:01.254094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:17:01.254094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:17:01.254094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:17:01.254094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:17:01.254094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:17:01.255041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:17:01.255041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:17:01.255041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:17:01.255041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:17:01.255041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:17:01.255041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:17:01.255041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:17:01.255041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:17:01.255041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:17:01.255041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:17:01.255041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:17:01.255041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:17:01.255041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:17:01.255041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:17:01.255041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:17:01.255041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:17:01.261053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:17:01.261053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:17:01.261053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:17:01.261053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:17:01.261053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:17:01.261053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:17:01.261053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:17:01.261053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:17:01.261053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:17:01.261053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:17:01.261053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:17:01.261053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:17:01.261053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 532/1079] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:17:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:17:01.261776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd21bd-8365-4dd3-be18-e0743260e634 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:01.312794 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:01.339585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a88664-443a-4f32-a8f7-8d0cd6f9582e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:01.353006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:01.362294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd21bd-8365-4dd3-be18-e0743260e634 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:01.375676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197860dd-1d16-4454-8988-751a3b3b1290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:01.376592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197860dd-1d16-4454-8988-751a3b3b1290 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:01.376875 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197860dd-1d16-4454-8988-751a3b3b1290 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:01.377145 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197860dd-1d16-4454-8988-751a3b3b1290 None admin] PATH_INFO: `/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/29048c2021e340b081ae44a390583e7a/roles/25bb48ece7784b5d9b0a60db78e81edd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:01.379347 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197860dd-1d16-4454-8988-751a3b3b1290 None admin] RBAC: Authorizing `identity:create_grant(project_id=e71a3acd0adf46a2a35b965a27c4998f, user_id=29048c2021e340b081ae44a390583e7a, role_id=25bb48ece7784b5d9b0a60db78e81edd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:01.380801 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197860dd-1d16-4454-8988-751a3b3b1290 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:01.393550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 548/1080] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:17:01 2022] PUT /identity/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/29048c2021e340b081ae44a390583e7a/roles/25bb48ece7784b5d9b0a60db78e81edd => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:17:01.396198 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6533a0-8f55-44ec-b0ec-793bd3dd6ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:01.396955 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6533a0-8f55-44ec-b0ec-793bd3dd6ebe None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:01.397236 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6533a0-8f55-44ec-b0ec-793bd3dd6ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:01.397516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6533a0-8f55-44ec-b0ec-793bd3dd6ebe None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:01.398273 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6533a0-8f55-44ec-b0ec-793bd3dd6ebe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:01.398586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6533a0-8f55-44ec-b0ec-793bd3dd6ebe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:01.399988 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6533a0-8f55-44ec-b0ec-793bd3dd6ebe None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:01.415047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 533/1081] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:17:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:01.416412 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197860dd-1d16-4454-8988-751a3b3b1290 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:01.521687 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197860dd-1d16-4454-8988-751a3b3b1290 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:01.556867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77cb0f-0ff1-430b-8e2b-7a6d82cef4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:01.557749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77cb0f-0ff1-430b-8e2b-7a6d82cef4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:01.558033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77cb0f-0ff1-430b-8e2b-7a6d82cef4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:01.558309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77cb0f-0ff1-430b-8e2b-7a6d82cef4d1 None admin] PATH_INFO: `/v3/users/7833119558c44d8bae729d0ad6a2f39b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:01.559515 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77cb0f-0ff1-430b-8e2b-7a6d82cef4d1 None admin] RBAC: Authorizing `identity:get_user(user_id=7833119558c44d8bae729d0ad6a2f39b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:01.560986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77cb0f-0ff1-430b-8e2b-7a6d82cef4d1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:01.562635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 549/1082] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:17:01 2022] GET /identity/v3/users/7833119558c44d8bae729d0ad6a2f39b => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:01.568816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6533a0-8f55-44ec-b0ec-793bd3dd6ebe None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:01.665417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6533a0-8f55-44ec-b0ec-793bd3dd6ebe None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:01.677602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:01.698260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a30b91-83d1-4f7a-abfd-6b723d61d694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:01.699111 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a30b91-83d1-4f7a-abfd-6b723d61d694 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:01.699391 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a30b91-83d1-4f7a-abfd-6b723d61d694 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:01.699664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a30b91-83d1-4f7a-abfd-6b723d61d694 None admin] PATH_INFO: `/v3/projects/438aa38c0c534fb9bbd18d9d4d4c3526` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:01.700857 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a30b91-83d1-4f7a-abfd-6b723d61d694 None admin] RBAC: Authorizing `identity:get_project(project_id=438aa38c0c534fb9bbd18d9d4d4c3526)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:01.702275 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a30b91-83d1-4f7a-abfd-6b723d61d694 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:01.703969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 534/1083] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:17:01 2022] GET /identity/v3/projects/438aa38c0c534fb9bbd18d9d4d4c3526 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:01.710763 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77cb0f-0ff1-430b-8e2b-7a6d82cef4d1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:01.752284 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:01.813651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77cb0f-0ff1-430b-8e2b-7a6d82cef4d1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:01.847772 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce3dc5-6bbf-4740-ba56-818a7df75e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:01.848697 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce3dc5-6bbf-4740-ba56-818a7df75e1d None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:01.848983 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce3dc5-6bbf-4740-ba56-818a7df75e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:01.849259 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce3dc5-6bbf-4740-ba56-818a7df75e1d None admin] PATH_INFO: `/v3/projects/438aa38c0c534fb9bbd18d9d4d4c3526/users/7833119558c44d8bae729d0ad6a2f39b/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:01.851805 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce3dc5-6bbf-4740-ba56-818a7df75e1d None admin] RBAC: Authorizing `identity:create_grant(project_id=438aa38c0c534fb9bbd18d9d4d4c3526, user_id=7833119558c44d8bae729d0ad6a2f39b, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:01.853272 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce3dc5-6bbf-4740-ba56-818a7df75e1d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:01.867219 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 550/1084] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:17:01 2022] PUT /identity/v3/projects/438aa38c0c534fb9bbd18d9d4d4c3526/users/7833119558c44d8bae729d0ad6a2f39b/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:17:03.397365 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139e45b-a741-4442-bfa3-d9e859f23016 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:03.397925 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139e45b-a741-4442-bfa3-d9e859f23016 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:03.397925 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139e45b-a741-4442-bfa3-d9e859f23016 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:03.399289 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 535/1085] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:17:03.404429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ccb49f-8b14-4cf9-8545-9dc79afeb167 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:03.404920 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ccb49f-8b14-4cf9-8545-9dc79afeb167 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:03.405040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ccb49f-8b14-4cf9-8545-9dc79afeb167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:03.421885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-76ccb49f-8b14-4cf9-8545-9dc79afeb167 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:03.423057 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-76ccb49f-8b14-4cf9-8545-9dc79afeb167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:03.488735 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 551/1086] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:17:03.511794 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139e45b-a741-4442-bfa3-d9e859f23016 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:03.612722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139e45b-a741-4442-bfa3-d9e859f23016 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:03.645942 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8da44-5a07-4bea-8c0e-7844c36738d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:03.647121 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8da44-5a07-4bea-8c0e-7844c36738d2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:03.647411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8da44-5a07-4bea-8c0e-7844c36738d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:03.647687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8da44-5a07-4bea-8c0e-7844c36738d2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:03.657969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f8da44-5a07-4bea-8c0e-7844c36738d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:17:03.666422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-92f8da44-5a07-4bea-8c0e-7844c36738d2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:17:03.666422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:03.666422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:17:03.666422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:17:03.666422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:17:03.666422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:17:03.666422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:17:03.666422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:17:03.666422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:17:03.666422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:17:03.666422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:17:03.666422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:17:03.666422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:17:03.666422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:17:03.666422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:17:03.666422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:17:03.667552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:17:03.667552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:17:03.667552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:17:03.667552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:17:03.667552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:17:03.667552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:17:03.667552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:17:03.667552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:17:03.667552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:17:03.667552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:17:03.667552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:17:03.667552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:17:03.667552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:17:03.667552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:17:03.667552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:17:03.667552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:17:03.668657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:17:03.668657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:17:03.668657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:17:03.668657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:17:03.668657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:17:03.668657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:17:03.668657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:17:03.668657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:17:03.668657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:17:03.668657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:17:03.668657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 10:17:03.668657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:17:03.672560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 536/1087] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Jul 18 10:17:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:17:03.673678 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ccb49f-8b14-4cf9-8545-9dc79afeb167 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:03.702630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:03.770106 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ccb49f-8b14-4cf9-8545-9dc79afeb167 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:03.803290 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116fb15-784c-4f96-80a4-e28c96ebdb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:03.803948 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116fb15-784c-4f96-80a4-e28c96ebdb1a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:03.804231 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116fb15-784c-4f96-80a4-e28c96ebdb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:03.804522 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116fb15-784c-4f96-80a4-e28c96ebdb1a None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:03.805267 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116fb15-784c-4f96-80a4-e28c96ebdb1a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:03.805571 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116fb15-784c-4f96-80a4-e28c96ebdb1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:03.806925 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116fb15-784c-4f96-80a4-e28c96ebdb1a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:03.814669 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 552/1088] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jul 18 10:17:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:03.820395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8da44-5a07-4bea-8c0e-7844c36738d2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:03.861728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99fded-d63d-49dd-b438-ac9e75b21a54 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:03.861939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:03.862080 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99fded-d63d-49dd-b438-ac9e75b21a54 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:03.862297 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99fded-d63d-49dd-b438-ac9e75b21a54 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:03.863685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 553/1089] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:17:03.868682 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64950c-2e0d-48a3-973a-179ad953ecce None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:03.868972 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64950c-2e0d-48a3-973a-179ad953ecce None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:03.869253 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64950c-2e0d-48a3-973a-179ad953ecce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:03.885838 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-be64950c-2e0d-48a3-973a-179ad953ecce None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:03.887020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-be64950c-2e0d-48a3-973a-179ad953ecce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:03.923276 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8da44-5a07-4bea-8c0e-7844c36738d2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:03.952535 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 554/1090] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:17:03.955083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d9df4-0626-4fe8-9b75-c0fd7b395746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:03.955867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d9df4-0626-4fe8-9b75-c0fd7b395746 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:03.956152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d9df4-0626-4fe8-9b75-c0fd7b395746 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:03.956427 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d9df4-0626-4fe8-9b75-c0fd7b395746 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:03.967673 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d9df4-0626-4fe8-9b75-c0fd7b395746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:03.969142 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d9df4-0626-4fe8-9b75-c0fd7b395746 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:03.976425 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64950c-2e0d-48a3-973a-179ad953ecce None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:03.986947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-147d9df4-0626-4fe8-9b75-c0fd7b395746 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:17:03.986947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:03.986947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:17:03.986947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:17:03.986947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:17:03.986947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:17:03.986947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:17:03.986947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:17:03.986947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:17:03.986947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:17:03.986947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:17:03.986947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:17:03.986947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:17:03.986947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:17:03.986947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:17:03.987993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:17:03.987993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:17:03.987993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:17:03.987993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:17:03.987993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:17:03.987993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:17:03.987993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:17:03.987993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:17:03.987993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:17:03.987993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:17:03.987993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:17:03.987993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:17:03.987993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:17:03.987993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:17:03.987993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:17:03.987993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:17:03.987993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:17:03.988932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:17:03.988932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:17:03.988932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:17:03.988932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:17:03.988932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 537/1091] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:17:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:17:03.993682 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d9df4-0626-4fe8-9b75-c0fd7b395746 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:04.003790 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:04.078012 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64950c-2e0d-48a3-973a-179ad953ecce None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:04.082431 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:04.092014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d9df4-0626-4fe8-9b75-c0fd7b395746 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:04.113537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb17f5-8cab-4c56-a529-a0554eb07575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:04.114320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb17f5-8cab-4c56-a529-a0554eb07575 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:04.114601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb17f5-8cab-4c56-a529-a0554eb07575 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:04.114875 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb17f5-8cab-4c56-a529-a0554eb07575 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:04.124717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb17f5-8cab-4c56-a529-a0554eb07575 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:17:04.125305 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a037d-4271-4cec-91a6-90c5ea65025f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:04.125959 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a037d-4271-4cec-91a6-90c5ea65025f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:04.126244 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a037d-4271-4cec-91a6-90c5ea65025f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:04.126520 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a037d-4271-4cec-91a6-90c5ea65025f None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:04.127242 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a037d-4271-4cec-91a6-90c5ea65025f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:04.127553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a037d-4271-4cec-91a6-90c5ea65025f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:04.128977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a037d-4271-4cec-91a6-90c5ea65025f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:04.132700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-dfbb17f5-8cab-4c56-a529-a0554eb07575 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:17:04.132700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:04.132700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:17:04.132700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:17:04.132700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:17:04.132700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:17:04.132700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:17:04.132700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:17:04.132700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:17:04.132700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:17:04.132700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:17:04.132700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:17:04.132700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:17:04.132700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:17:04.132700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:17:04.132700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:17:04.133849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:17:04.133849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:17:04.133849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:17:04.133849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:17:04.133849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:17:04.133849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:17:04.133849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:17:04.133849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:17:04.133849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:17:04.133849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:17:04.133849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:17:04.133849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:17:04.133849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:17:04.133849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:17:04.133849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:17:04.133849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:17:04.134843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:17:04.134843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:17:04.134843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:17:04.134843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:17:04.134843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:17:04.134843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:17:04.134843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:17:04.134843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:17:04.134843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:17:04.134843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:17:04.134843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:17:04.134843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:17:04.134843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 555/1092] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:17:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:17:04.139765 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb17f5-8cab-4c56-a529-a0554eb07575 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:04.146349 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 538/1093] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:17:04.147794 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a037d-4271-4cec-91a6-90c5ea65025f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:04.225735 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:04.243896 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb17f5-8cab-4c56-a529-a0554eb07575 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:04.248248 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a037d-4271-4cec-91a6-90c5ea65025f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:04.278836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a946ede-8409-48ac-84af-117a5fb92950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:04.279559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a946ede-8409-48ac-84af-117a5fb92950 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:04.279850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a946ede-8409-48ac-84af-117a5fb92950 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:04.280130 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a946ede-8409-48ac-84af-117a5fb92950 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:04.280915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a946ede-8409-48ac-84af-117a5fb92950 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:04.281224 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a946ede-8409-48ac-84af-117a5fb92950 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:04.282056 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3ac35-dba2-4ef1-97a4-e15e7b2c42ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:04.282601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a946ede-8409-48ac-84af-117a5fb92950 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:04.282842 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3ac35-dba2-4ef1-97a4-e15e7b2c42ac None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:04.283126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3ac35-dba2-4ef1-97a4-e15e7b2c42ac None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:04.283402 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3ac35-dba2-4ef1-97a4-e15e7b2c42ac None admin] PATH_INFO: `/v3/users/nova` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:04.290390 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3ac35-dba2-4ef1-97a4-e15e7b2c42ac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:04.290824 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 556/1094] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:17:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:17:04.291848 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3ac35-dba2-4ef1-97a4-e15e7b2c42ac None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:04.296488 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a946ede-8409-48ac-84af-117a5fb92950 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:04.306171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e8d3ac35-dba2-4ef1-97a4-e15e7b2c42ac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:17:04.306171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:04.306171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:17:04.306171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:17:04.306171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:17:04.306171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:17:04.306171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:17:04.306171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:17:04.306171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:17:04.306171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:17:04.306171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:17:04.306171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:17:04.306171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:17:04.306171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:17:04.306171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:17:04.306171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:17:04.307152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:17:04.307152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:17:04.307152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:17:04.307152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:17:04.307152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:17:04.307152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:17:04.307152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:17:04.307152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:17:04.307152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:17:04.307152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:17:04.307152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:17:04.307152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:17:04.307152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:17:04.307152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:17:04.307152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:17:04.307152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:17:04.307152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:17:04.307152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:17:04.308468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:17:04.308468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:17:04.308468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:17:04.308468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:17:04.308468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:17:04.308468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:17:04.308468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:17:04.308468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:17:04.308468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 10:17:04.308468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:17:04.308468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 539/1095] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jul 18 10:17:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:17:04.312980 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3ac35-dba2-4ef1-97a4-e15e7b2c42ac None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:04.327821 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:04.395679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a946ede-8409-48ac-84af-117a5fb92950 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:04.412561 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3ac35-dba2-4ef1-97a4-e15e7b2c42ac None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:04.429070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8e0ed-1f4e-44df-98e2-4ac1fb2560d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:04.429939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8e0ed-1f4e-44df-98e2-4ac1fb2560d8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:04.430225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8e0ed-1f4e-44df-98e2-4ac1fb2560d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:04.430505 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8e0ed-1f4e-44df-98e2-4ac1fb2560d8 None admin] PATH_INFO: `/v3/users/7833119558c44d8bae729d0ad6a2f39b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:04.431698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8e0ed-1f4e-44df-98e2-4ac1fb2560d8 None admin] RBAC: Authorizing `identity:get_user(user_id=7833119558c44d8bae729d0ad6a2f39b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:04.433183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8e0ed-1f4e-44df-98e2-4ac1fb2560d8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:04.434831 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 557/1096] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:17:04 2022] GET /identity/v3/users/7833119558c44d8bae729d0ad6a2f39b => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:17:04.440478 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8e0ed-1f4e-44df-98e2-4ac1fb2560d8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:04.445941 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811653c-2961-49dd-95a5-b444c4813f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:04.446606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811653c-2961-49dd-95a5-b444c4813f81 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:04.446890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811653c-2961-49dd-95a5-b444c4813f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:04.447166 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811653c-2961-49dd-95a5-b444c4813f81 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:04.448008 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811653c-2961-49dd-95a5-b444c4813f81 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:04.448322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811653c-2961-49dd-95a5-b444c4813f81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:04.449790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811653c-2961-49dd-95a5-b444c4813f81 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:04.455524 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:04.467267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 540/1097] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jul 18 10:17:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:04.472856 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811653c-2961-49dd-95a5-b444c4813f81 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:04.541279 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8e0ed-1f4e-44df-98e2-4ac1fb2560d8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:04.542089 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:04.572226 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811653c-2961-49dd-95a5-b444c4813f81 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:04.574022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f54153-f5f9-4b85-bdd2-b121d4ca3fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:04.575386 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f54153-f5f9-4b85-bdd2-b121d4ca3fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:04.575669 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f54153-f5f9-4b85-bdd2-b121d4ca3fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:04.575944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f54153-f5f9-4b85-bdd2-b121d4ca3fb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:04.606688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44df39-453d-42fd-bb1e-dd5dcddac776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:04.607313 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44df39-453d-42fd-bb1e-dd5dcddac776 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:04.607601 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44df39-453d-42fd-bb1e-dd5dcddac776 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:04.607881 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44df39-453d-42fd-bb1e-dd5dcddac776 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:04.645421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f54153-f5f9-4b85-bdd2-b121d4ca3fb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:04.647415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f54153-f5f9-4b85-bdd2-b121d4ca3fb1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:04.661101 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:04.676943 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44df39-453d-42fd-bb1e-dd5dcddac776 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:04.678414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44df39-453d-42fd-bb1e-dd5dcddac776 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:04.743921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:04.745527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 558/1098] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:17:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 10:17:04.751359 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f54153-f5f9-4b85-bdd2-b121d4ca3fb1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:04.775249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 541/1099] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:17:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 10:17:04.780974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44df39-453d-42fd-bb1e-dd5dcddac776 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:04.854807 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f54153-f5f9-4b85-bdd2-b121d4ca3fb1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:04.868867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:04.877689 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44df39-453d-42fd-bb1e-dd5dcddac776 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:04.890722 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a0776-8da7-468d-9270-208ecf72377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:04.891576 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a0776-8da7-468d-9270-208ecf72377f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:04.891869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a0776-8da7-468d-9270-208ecf72377f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:04.892147 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a0776-8da7-468d-9270-208ecf72377f None admin] PATH_INFO: `/v3/projects/438aa38c0c534fb9bbd18d9d4d4c3526` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:04.893357 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a0776-8da7-468d-9270-208ecf72377f None admin] RBAC: Authorizing `identity:get_project(project_id=438aa38c0c534fb9bbd18d9d4d4c3526)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:04.894795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a0776-8da7-468d-9270-208ecf72377f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:04.896522 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 559/1100] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:17:04 2022] GET /identity/v3/projects/438aa38c0c534fb9bbd18d9d4d4c3526 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:17:04.902431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a0776-8da7-468d-9270-208ecf72377f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:04.911006 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194c953-5db9-4a3a-b2bc-3ce5bbf22395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:04.911791 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194c953-5db9-4a3a-b2bc-3ce5bbf22395 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:04.912076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194c953-5db9-4a3a-b2bc-3ce5bbf22395 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:04.912914 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194c953-5db9-4a3a-b2bc-3ce5bbf22395 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:04.922781 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194c953-5db9-4a3a-b2bc-3ce5bbf22395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:04.924216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194c953-5db9-4a3a-b2bc-3ce5bbf22395 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:04.941734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-1194c953-5db9-4a3a-b2bc-3ce5bbf22395 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:17:04.941734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:04.941734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:17:04.941734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:17:04.941734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:17:04.941734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:17:04.941734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:17:04.941734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:17:04.941734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:17:04.941734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:17:04.941734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:17:04.941734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:17:04.941734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:17:04.941734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:17:04.941734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:17:04.942723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:17:04.942723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:17:04.942723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:17:04.942723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:17:04.942723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:17:04.942723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:17:04.942723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:17:04.942723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:17:04.942723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:17:04.942723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:17:04.942723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:17:04.942723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:17:04.942723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:17:04.942723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:17:04.942723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:17:04.942723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:17:04.942723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:17:04.944326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:17:04.944326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:17:04.944326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:17:04.944326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:17:04.944326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 542/1101] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:17:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:17:04.948976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194c953-5db9-4a3a-b2bc-3ce5bbf22395 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:04.973509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:05.002513 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a0776-8da7-468d-9270-208ecf72377f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:05.039012 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717bc1b8-c607-41f2-8534-019172dbf12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:05.039654 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bc1b8-c607-41f2-8534-019172dbf12c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:05.039933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bc1b8-c607-41f2-8534-019172dbf12c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:05.040204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bc1b8-c607-41f2-8534-019172dbf12c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:05.041060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717bc1b8-c607-41f2-8534-019172dbf12c None admin] RBAC: Adding query filter params (role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.project.id=438aa38c0c534fb9bbd18d9d4d4c3526, user.id=7833119558c44d8bae729d0ad6a2f39b) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:05.041368 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717bc1b8-c607-41f2-8534-019172dbf12c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:05.042790 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717bc1b8-c607-41f2-8534-019172dbf12c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:05.060555 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 560/1102] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:17:04 2022] GET /identity/v3/role_assignments?user.id=7833119558c44d8bae729d0ad6a2f39b&scope.project.id=438aa38c0c534fb9bbd18d9d4d4c3526&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:17:05.060787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194c953-5db9-4a3a-b2bc-3ce5bbf22395 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:05.089118 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f68444-f960-4216-8b59-080691edabf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:05.089790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f68444-f960-4216-8b59-080691edabf6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:05.090076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f68444-f960-4216-8b59-080691edabf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:05.090356 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f68444-f960-4216-8b59-080691edabf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:05.091093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f68444-f960-4216-8b59-080691edabf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:05.091403 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f68444-f960-4216-8b59-080691edabf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:05.092818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f68444-f960-4216-8b59-080691edabf6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:05.110375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 543/1103] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:17:05.111514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717bc1b8-c607-41f2-8534-019172dbf12c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:05.160759 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:05.218776 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717bc1b8-c607-41f2-8534-019172dbf12c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:05.253948 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f2540d-bacd-4ebf-b66a-f7e8fbacc34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:05.254742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2540d-bacd-4ebf-b66a-f7e8fbacc34a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:05.255023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2540d-bacd-4ebf-b66a-f7e8fbacc34a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:05.255294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2540d-bacd-4ebf-b66a-f7e8fbacc34a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:05.261973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f2540d-bacd-4ebf-b66a-f7e8fbacc34a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:05.263431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f2540d-bacd-4ebf-b66a-f7e8fbacc34a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:05.278316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-46f2540d-bacd-4ebf-b66a-f7e8fbacc34a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:17:05.278316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:05.278316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:17:05.278316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:17:05.278316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:17:05.278316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:17:05.278316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:17:05.278316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:17:05.278316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:17:05.278316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:17:05.278316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:17:05.278316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:17:05.278316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:17:05.278316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:17:05.278316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:17:05.279421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:17:05.279421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:17:05.279421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:17:05.279421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:17:05.279421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:17:05.279421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:17:05.279421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:17:05.279421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:17:05.279421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:17:05.279421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:17:05.279421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:17:05.279421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:17:05.279421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:17:05.279421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:17:05.279421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:17:05.279421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:17:05.279421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:17:05.280372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:17:05.280372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:17:05.280372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:17:05.280372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:17:05.280372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:17:05.280372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:17:05.280372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:17:05.280372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:17:05.280372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 561/1104] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:17:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:17:05.285264 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f68444-f960-4216-8b59-080691edabf6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:05.364302 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:05.382331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f68444-f960-4216-8b59-080691edabf6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:05.415558 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470e2a6-e286-4ec2-9f51-f52d45c2606c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:05.416203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470e2a6-e286-4ec2-9f51-f52d45c2606c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:05.416484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470e2a6-e286-4ec2-9f51-f52d45c2606c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:05.416784 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470e2a6-e286-4ec2-9f51-f52d45c2606c None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:05.417565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6470e2a6-e286-4ec2-9f51-f52d45c2606c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:05.417875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6470e2a6-e286-4ec2-9f51-f52d45c2606c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:05.419291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6470e2a6-e286-4ec2-9f51-f52d45c2606c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:05.431193 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 544/1105] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:17:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:05.437216 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f2540d-bacd-4ebf-b66a-f7e8fbacc34a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:05.533087 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f2540d-bacd-4ebf-b66a-f7e8fbacc34a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:05.535390 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:05.567003 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41325df8-e7b9-4e0f-8217-1bc250c338ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:05.567648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41325df8-e7b9-4e0f-8217-1bc250c338ff None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:05.567934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41325df8-e7b9-4e0f-8217-1bc250c338ff None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:05.568210 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41325df8-e7b9-4e0f-8217-1bc250c338ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:05.569106 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41325df8-e7b9-4e0f-8217-1bc250c338ff None admin] RBAC: Adding query filter params (role.id=25bb48ece7784b5d9b0a60db78e81edd, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=29048c2021e340b081ae44a390583e7a) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:05.569416 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41325df8-e7b9-4e0f-8217-1bc250c338ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:05.570838 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41325df8-e7b9-4e0f-8217-1bc250c338ff None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:05.576522 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 562/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:17:05 2022] GET /identity/v3/role_assignments?user.id=29048c2021e340b081ae44a390583e7a&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=25bb48ece7784b5d9b0a60db78e81edd => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:17:07.074454 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af7b3e-f17f-4db6-83ff-e6c1b749afa8 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:07.074979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af7b3e-f17f-4db6-83ff-e6c1b749afa8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:07.075075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af7b3e-f17f-4db6-83ff-e6c1b749afa8 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:07.076373 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 545/1107] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:17:07.081593 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f35f96-a0b7-4d15-9fbf-8d5058a68bb8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:07.081990 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f35f96-a0b7-4d15-9fbf-8d5058a68bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:07.082170 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f35f96-a0b7-4d15-9fbf-8d5058a68bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:07.098684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-82f35f96-a0b7-4d15-9fbf-8d5058a68bb8 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:07.099910 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-82f35f96-a0b7-4d15-9fbf-8d5058a68bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:07.165880 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 563/1108] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:17:07.190162 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af7b3e-f17f-4db6-83ff-e6c1b749afa8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:07.287869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af7b3e-f17f-4db6-83ff-e6c1b749afa8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:07.320191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca0fcc-234f-4ef0-a794-d66dced65bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:07.320994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca0fcc-234f-4ef0-a794-d66dced65bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:07.321281 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca0fcc-234f-4ef0-a794-d66dced65bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:07.321557 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca0fcc-234f-4ef0-a794-d66dced65bb8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:07.322717 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca0fcc-234f-4ef0-a794-d66dced65bb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:07.324152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca0fcc-234f-4ef0-a794-d66dced65bb8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:07.325846 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 546/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:17:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:07.331586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f35f96-a0b7-4d15-9fbf-8d5058a68bb8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:07.423952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:07.428623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f35f96-a0b7-4d15-9fbf-8d5058a68bb8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:07.461153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35847f91-544f-4e03-8276-e04ff75c9471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:07.461801 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35847f91-544f-4e03-8276-e04ff75c9471 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:07.462081 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35847f91-544f-4e03-8276-e04ff75c9471 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:07.462351 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35847f91-544f-4e03-8276-e04ff75c9471 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:07.463048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35847f91-544f-4e03-8276-e04ff75c9471 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:07.464915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35847f91-544f-4e03-8276-e04ff75c9471 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:07.484628 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 564/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 18 10:17:07 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:17:09.505063 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7a104-4e0c-4475-a3da-0510facca595 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:09.505613 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7a104-4e0c-4475-a3da-0510facca595 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:09.505613 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7a104-4e0c-4475-a3da-0510facca595 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:09.507998 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 547/1111] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:17:09.513197 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b2891-744b-4f84-ad33-2dc971ae966b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:09.513600 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b2891-744b-4f84-ad33-2dc971ae966b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:09.513758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b2891-744b-4f84-ad33-2dc971ae966b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:09.530700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0d7b2891-744b-4f84-ad33-2dc971ae966b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:09.531879 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0d7b2891-744b-4f84-ad33-2dc971ae966b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:09.595812 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 565/1112] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:17:09.618912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7a104-4e0c-4475-a3da-0510facca595 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:09.631425 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:09.717396 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7a104-4e0c-4475-a3da-0510facca595 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:09.749495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291565b-5bc7-4446-9f69-0d3d719160b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:09.750338 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291565b-5bc7-4446-9f69-0d3d719160b1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:09.750622 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291565b-5bc7-4446-9f69-0d3d719160b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:09.750899 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291565b-5bc7-4446-9f69-0d3d719160b1 None admin] PATH_INFO: `/v3/roles/8c039234db9640d0bdc0c2a629ff9e52` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:09.752357 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291565b-5bc7-4446-9f69-0d3d719160b1 None admin] RBAC: Authorizing `identity:get_role(role_id=8c039234db9640d0bdc0c2a629ff9e52)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:09.753746 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291565b-5bc7-4446-9f69-0d3d719160b1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:09.755153 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 548/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:17:09 2022] GET /identity/v3/roles/8c039234db9640d0bdc0c2a629ff9e52 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:09.761405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b2891-744b-4f84-ad33-2dc971ae966b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:09.841258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:09.859108 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b2891-744b-4f84-ad33-2dc971ae966b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:09.899353 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23edd877-e71c-41f5-91e5-8d14cba4ef12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:09.900219 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edd877-e71c-41f5-91e5-8d14cba4ef12 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:09.900532 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edd877-e71c-41f5-91e5-8d14cba4ef12 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:09.900810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edd877-e71c-41f5-91e5-8d14cba4ef12 None admin] PATH_INFO: `/v3/users/2dc2246337694782a885a0696637c305` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:09.913668 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23edd877-e71c-41f5-91e5-8d14cba4ef12 None admin] RBAC: Authorizing `identity:get_user(user_id=2dc2246337694782a885a0696637c305)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:09.915115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23edd877-e71c-41f5-91e5-8d14cba4ef12 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:09.916821 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 566/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:17:09 2022] GET /identity/v3/users/2dc2246337694782a885a0696637c305 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:09.922583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291565b-5bc7-4446-9f69-0d3d719160b1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:09.999840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:10.020786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291565b-5bc7-4446-9f69-0d3d719160b1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:10.053632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684452e8-d515-4a5f-bca8-3d8211ad4055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:10.054260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684452e8-d515-4a5f-bca8-3d8211ad4055 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:10.054541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684452e8-d515-4a5f-bca8-3d8211ad4055 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:10.054815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684452e8-d515-4a5f-bca8-3d8211ad4055 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:10.118602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684452e8-d515-4a5f-bca8-3d8211ad4055 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:10.120031 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684452e8-d515-4a5f-bca8-3d8211ad4055 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:10.199934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:10.221666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 549/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:17:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:17:10.228029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23edd877-e71c-41f5-91e5-8d14cba4ef12 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:10.323859 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23edd877-e71c-41f5-91e5-8d14cba4ef12 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:10.356434 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c4081-dc33-4af0-9795-a3dc798d477c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:10.357305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c4081-dc33-4af0-9795-a3dc798d477c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:10.357587 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c4081-dc33-4af0-9795-a3dc798d477c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:10.357858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c4081-dc33-4af0-9795-a3dc798d477c None admin] PATH_INFO: `/v3/projects/438aa38c0c534fb9bbd18d9d4d4c3526` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:10.359045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c4081-dc33-4af0-9795-a3dc798d477c None admin] RBAC: Authorizing `identity:get_project(project_id=438aa38c0c534fb9bbd18d9d4d4c3526)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:10.360470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c4081-dc33-4af0-9795-a3dc798d477c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:10.362169 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 567/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:17:10 2022] GET /identity/v3/projects/438aa38c0c534fb9bbd18d9d4d4c3526 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:10.368209 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684452e8-d515-4a5f-bca8-3d8211ad4055 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:10.464096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684452e8-d515-4a5f-bca8-3d8211ad4055 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:10.496012 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67daa66-88d1-4d6c-ba33-0faedfd13c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:10.496675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67daa66-88d1-4d6c-ba33-0faedfd13c8d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:10.496960 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67daa66-88d1-4d6c-ba33-0faedfd13c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:10.497235 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67daa66-88d1-4d6c-ba33-0faedfd13c8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:10.498055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67daa66-88d1-4d6c-ba33-0faedfd13c8d None admin] RBAC: Adding query filter params (role.id=8c039234db9640d0bdc0c2a629ff9e52, scope.project.id=438aa38c0c534fb9bbd18d9d4d4c3526, user.id=2dc2246337694782a885a0696637c305) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:10.498364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67daa66-88d1-4d6c-ba33-0faedfd13c8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:10.499778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67daa66-88d1-4d6c-ba33-0faedfd13c8d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:10.504986 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 550/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:17:10 2022] GET /identity/v3/role_assignments?user.id=2dc2246337694782a885a0696637c305&scope.project.id=438aa38c0c534fb9bbd18d9d4d4c3526&role.id=8c039234db9640d0bdc0c2a629ff9e52 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:12.449657 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b97dee-9f95-4ae4-967d-5bfaf575fade None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:12.450174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b97dee-9f95-4ae4-967d-5bfaf575fade None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:12.450174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b97dee-9f95-4ae4-967d-5bfaf575fade None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:12.451510 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 568/1118] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:17:12.456565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f46ef-08f0-4c73-8757-b600acc84378 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:12.456994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f46ef-08f0-4c73-8757-b600acc84378 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:12.457131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f46ef-08f0-4c73-8757-b600acc84378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:12.474223 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-419f46ef-08f0-4c73-8757-b600acc84378 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:12.475421 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-419f46ef-08f0-4c73-8757-b600acc84378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:12.509381 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:12.539916 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 551/1119] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:17:12.563981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b97dee-9f95-4ae4-967d-5bfaf575fade None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:12.576753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:12.663210 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b97dee-9f95-4ae4-967d-5bfaf575fade None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:12.695593 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b41bb-1026-48af-96bc-dc1f0410ffdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:12.696424 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b41bb-1026-48af-96bc-dc1f0410ffdb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:12.696736 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b41bb-1026-48af-96bc-dc1f0410ffdb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:12.697014 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b41bb-1026-48af-96bc-dc1f0410ffdb None admin] PATH_INFO: `/v3/roles/8c039234db9640d0bdc0c2a629ff9e52` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:12.698479 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b41bb-1026-48af-96bc-dc1f0410ffdb None admin] RBAC: Authorizing `identity:get_role(role_id=8c039234db9640d0bdc0c2a629ff9e52)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:12.699886 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b41bb-1026-48af-96bc-dc1f0410ffdb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:12.701238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 569/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:17:12 2022] GET /identity/v3/roles/8c039234db9640d0bdc0c2a629ff9e52 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:12.707103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f46ef-08f0-4c73-8757-b600acc84378 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:12.803679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f46ef-08f0-4c73-8757-b600acc84378 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:12.836287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5aacc-d080-46e0-b1a0-825c22fc2678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:12.837168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5aacc-d080-46e0-b1a0-825c22fc2678 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:12.837456 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5aacc-d080-46e0-b1a0-825c22fc2678 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:12.837737 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5aacc-d080-46e0-b1a0-825c22fc2678 None admin] PATH_INFO: `/v3/users/2dc2246337694782a885a0696637c305` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:12.838915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5aacc-d080-46e0-b1a0-825c22fc2678 None admin] RBAC: Authorizing `identity:get_user(user_id=2dc2246337694782a885a0696637c305)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:12.840341 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e5aacc-d080-46e0-b1a0-825c22fc2678 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:12.841986 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 552/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:17:12 2022] GET /identity/v3/users/2dc2246337694782a885a0696637c305 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:12.847723 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b41bb-1026-48af-96bc-dc1f0410ffdb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:12.924812 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:12.946856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b41bb-1026-48af-96bc-dc1f0410ffdb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:12.978995 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9922199-f37e-4ca8-b7c4-32ca1fae15e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:12.979783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9922199-f37e-4ca8-b7c4-32ca1fae15e4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:12.980064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9922199-f37e-4ca8-b7c4-32ca1fae15e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:12.980333 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9922199-f37e-4ca8-b7c4-32ca1fae15e4 None admin] PATH_INFO: `/v3/projects/438aa38c0c534fb9bbd18d9d4d4c3526` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:12.981782 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9922199-f37e-4ca8-b7c4-32ca1fae15e4 None admin] RBAC: Authorizing `identity:get_project(project_id=438aa38c0c534fb9bbd18d9d4d4c3526)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:12.983239 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9922199-f37e-4ca8-b7c4-32ca1fae15e4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:12.984955 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 570/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:17:12 2022] GET /identity/v3/projects/438aa38c0c534fb9bbd18d9d4d4c3526 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:12.991483 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5aacc-d080-46e0-b1a0-825c22fc2678 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:13.031784 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:13.087357 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5aacc-d080-46e0-b1a0-825c22fc2678 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:13.125801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191531c-9e5d-4ed3-b097-c210ff935506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:13.126677 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191531c-9e5d-4ed3-b097-c210ff935506 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:13.126958 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191531c-9e5d-4ed3-b097-c210ff935506 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:13.127235 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191531c-9e5d-4ed3-b097-c210ff935506 None admin] PATH_INFO: `/v3/projects/438aa38c0c534fb9bbd18d9d4d4c3526/users/2dc2246337694782a885a0696637c305/roles/8c039234db9640d0bdc0c2a629ff9e52` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:13.129423 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191531c-9e5d-4ed3-b097-c210ff935506 None admin] RBAC: Authorizing `identity:create_grant(project_id=438aa38c0c534fb9bbd18d9d4d4c3526, user_id=2dc2246337694782a885a0696637c305, role_id=8c039234db9640d0bdc0c2a629ff9e52)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:13.131433 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191531c-9e5d-4ed3-b097-c210ff935506 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:13.143459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 553/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:17:12 2022] PUT /identity/v3/projects/438aa38c0c534fb9bbd18d9d4d4c3526/users/2dc2246337694782a885a0696637c305/roles/8c039234db9640d0bdc0c2a629ff9e52 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:17:15.106470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0a7c7-4412-43e0-ad56-4576ba1890b2 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:15.107029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0a7c7-4412-43e0-ad56-4576ba1890b2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:15.107029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0a7c7-4412-43e0-ad56-4576ba1890b2 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:15.108423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 571/1124] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:17:15.113763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f60230-cca8-4848-95cc-100a7ae752cb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:15.114201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f60230-cca8-4848-95cc-100a7ae752cb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:15.114329 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f60230-cca8-4848-95cc-100a7ae752cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:15.137058 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-42f60230-cca8-4848-95cc-100a7ae752cb None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:15.138305 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-42f60230-cca8-4848-95cc-100a7ae752cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:15.202530 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 554/1125] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:17:15.226241 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0a7c7-4412-43e0-ad56-4576ba1890b2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:15.326383 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0a7c7-4412-43e0-ad56-4576ba1890b2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:15.358328 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec210e-50d7-484b-a80d-1b4d21b86470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:15.359104 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec210e-50d7-484b-a80d-1b4d21b86470 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:15.359385 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec210e-50d7-484b-a80d-1b4d21b86470 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:15.359659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec210e-50d7-484b-a80d-1b4d21b86470 None admin] PATH_INFO: `/v3/roles/8c039234db9640d0bdc0c2a629ff9e52` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:15.361611 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec210e-50d7-484b-a80d-1b4d21b86470 None admin] RBAC: Authorizing `identity:get_role(role_id=8c039234db9640d0bdc0c2a629ff9e52)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:15.363078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec210e-50d7-484b-a80d-1b4d21b86470 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:15.364631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 572/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:17:15 2022] GET /identity/v3/roles/8c039234db9640d0bdc0c2a629ff9e52 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:15.370995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f60230-cca8-4848-95cc-100a7ae752cb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:15.399854 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:15.467133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f60230-cca8-4848-95cc-100a7ae752cb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:15.499886 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f738b3-5ce2-4964-8ea6-6e3705141dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:15.500761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f738b3-5ce2-4964-8ea6-6e3705141dce None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:15.501045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f738b3-5ce2-4964-8ea6-6e3705141dce None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:15.501320 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f738b3-5ce2-4964-8ea6-6e3705141dce None admin] PATH_INFO: `/v3/users/2dc2246337694782a885a0696637c305` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:15.502722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f738b3-5ce2-4964-8ea6-6e3705141dce None admin] RBAC: Authorizing `identity:get_user(user_id=2dc2246337694782a885a0696637c305)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:15.504141 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f738b3-5ce2-4964-8ea6-6e3705141dce None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:15.505809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 555/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:17:15 2022] GET /identity/v3/users/2dc2246337694782a885a0696637c305 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:15.511635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec210e-50d7-484b-a80d-1b4d21b86470 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:15.530980 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:15.612333 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec210e-50d7-484b-a80d-1b4d21b86470 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:15.644960 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a5d0c-fbb1-4f13-8171-a3a612a344d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:15.645599 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a5d0c-fbb1-4f13-8171-a3a612a344d1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:15.645883 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a5d0c-fbb1-4f13-8171-a3a612a344d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:15.646160 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a5d0c-fbb1-4f13-8171-a3a612a344d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:15.709954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a5d0c-fbb1-4f13-8171-a3a612a344d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:15.711390 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a5d0c-fbb1-4f13-8171-a3a612a344d1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:15.725435 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:15.806332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 573/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:17:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:17:15.812210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f738b3-5ce2-4964-8ea6-6e3705141dce None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:15.909916 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:15.914025 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f738b3-5ce2-4964-8ea6-6e3705141dce None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:15.948460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c037a-7df3-4d47-8d16-34967f3e2a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:15.949341 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c037a-7df3-4d47-8d16-34967f3e2a9e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:15.949626 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c037a-7df3-4d47-8d16-34967f3e2a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:15.949906 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c037a-7df3-4d47-8d16-34967f3e2a9e None admin] PATH_INFO: `/v3/projects/438aa38c0c534fb9bbd18d9d4d4c3526` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:15.951065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c037a-7df3-4d47-8d16-34967f3e2a9e None admin] RBAC: Authorizing `identity:get_project(project_id=438aa38c0c534fb9bbd18d9d4d4c3526)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:15.952497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c037a-7df3-4d47-8d16-34967f3e2a9e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:15.954210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 556/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:17:15 2022] GET /identity/v3/projects/438aa38c0c534fb9bbd18d9d4d4c3526 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:15.960332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a5d0c-fbb1-4f13-8171-a3a612a344d1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:16.053776 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a5d0c-fbb1-4f13-8171-a3a612a344d1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:16.066950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:16.086708 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec511d-49e1-4d49-9816-98a706508e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:16.087343 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec511d-49e1-4d49-9816-98a706508e04 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:16.087625 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec511d-49e1-4d49-9816-98a706508e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:16.087911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec511d-49e1-4d49-9816-98a706508e04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:16.088749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec511d-49e1-4d49-9816-98a706508e04 None admin] RBAC: Adding query filter params (role.id=8c039234db9640d0bdc0c2a629ff9e52, scope.project.id=438aa38c0c534fb9bbd18d9d4d4c3526, user.id=2dc2246337694782a885a0696637c305) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:16.089060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec511d-49e1-4d49-9816-98a706508e04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:16.090475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec511d-49e1-4d49-9816-98a706508e04 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:16.096061 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 574/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:17:15 2022] GET /identity/v3/role_assignments?user.id=2dc2246337694782a885a0696637c305&scope.project.id=438aa38c0c534fb9bbd18d9d4d4c3526&role.id=8c039234db9640d0bdc0c2a629ff9e52 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:18.085085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c163452-c19b-49da-97d0-5125c840ca73 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:18.085642 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c163452-c19b-49da-97d0-5125c840ca73 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:18.085642 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c163452-c19b-49da-97d0-5125c840ca73 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:18.087016 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 557/1131] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:17:18.092057 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebb8e5-a681-4702-aba3-4946abecc64c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:18.092472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebb8e5-a681-4702-aba3-4946abecc64c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:18.092643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebb8e5-a681-4702-aba3-4946abecc64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:18.109719 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-36ebb8e5-a681-4702-aba3-4946abecc64c None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:18.110887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-36ebb8e5-a681-4702-aba3-4946abecc64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:18.174792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 575/1132] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:17:18.197251 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c163452-c19b-49da-97d0-5125c840ca73 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:18.294016 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c163452-c19b-49da-97d0-5125c840ca73 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:18.325409 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1aaed2-eb11-498d-9a12-fd8aff9e97e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:18.326177 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1aaed2-eb11-498d-9a12-fd8aff9e97e3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:18.326459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1aaed2-eb11-498d-9a12-fd8aff9e97e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:18.326730 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1aaed2-eb11-498d-9a12-fd8aff9e97e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:18.327877 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1aaed2-eb11-498d-9a12-fd8aff9e97e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:18.329310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1aaed2-eb11-498d-9a12-fd8aff9e97e3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:18.330895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 558/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:17:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:18.337373 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ebb8e5-a681-4702-aba3-4946abecc64c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:18.432095 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ebb8e5-a681-4702-aba3-4946abecc64c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:18.441599 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:18.466377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90390fb-70ca-4ad2-9525-a058ad936fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:18.467020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90390fb-70ca-4ad2-9525-a058ad936fdf None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:18.467304 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90390fb-70ca-4ad2-9525-a058ad936fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:18.467582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90390fb-70ca-4ad2-9525-a058ad936fdf None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:18.468286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90390fb-70ca-4ad2-9525-a058ad936fdf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:18.469740 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90390fb-70ca-4ad2-9525-a058ad936fdf None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:18.498214 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 576/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 18 10:17:18 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:17:20.499869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec409cc8-bcdc-4355-81f6-52e96a5aca06 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:20.500372 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec409cc8-bcdc-4355-81f6-52e96a5aca06 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:20.500372 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec409cc8-bcdc-4355-81f6-52e96a5aca06 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:20.501740 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 559/1135] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:17:20.506963 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc991723-cac7-4bd6-a2c2-d1e2bc234c98 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:20.507383 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc991723-cac7-4bd6-a2c2-d1e2bc234c98 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:20.507530 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc991723-cac7-4bd6-a2c2-d1e2bc234c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:20.524415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fc991723-cac7-4bd6-a2c2-d1e2bc234c98 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:20.525620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fc991723-cac7-4bd6-a2c2-d1e2bc234c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:20.598062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 577/1136] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:17:20.623435 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec409cc8-bcdc-4355-81f6-52e96a5aca06 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:20.657865 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:20.721852 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec409cc8-bcdc-4355-81f6-52e96a5aca06 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:20.754057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacbbf5-31c5-429b-9042-ffd4c2693440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:20.754775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacbbf5-31c5-429b-9042-ffd4c2693440 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:20.755060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacbbf5-31c5-429b-9042-ffd4c2693440 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:20.755336 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacbbf5-31c5-429b-9042-ffd4c2693440 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:20.757036 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeacbbf5-31c5-429b-9042-ffd4c2693440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:20.758463 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeacbbf5-31c5-429b-9042-ffd4c2693440 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:20.760056 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 560/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:17:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:20.765885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc991723-cac7-4bd6-a2c2-d1e2bc234c98 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:20.853337 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:20.863593 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc991723-cac7-4bd6-a2c2-d1e2bc234c98 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:20.896190 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac844b-70fc-4060-8a94-dbfb0ca6d15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:20.897455 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac844b-70fc-4060-8a94-dbfb0ca6d15f None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:20.897740 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac844b-70fc-4060-8a94-dbfb0ca6d15f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:20.898015 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac844b-70fc-4060-8a94-dbfb0ca6d15f None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:20.898719 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac844b-70fc-4060-8a94-dbfb0ca6d15f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:20.900144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac844b-70fc-4060-8a94-dbfb0ca6d15f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:20.918929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 578/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 18 10:17:20 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:17:22.950799 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d94797-ca95-4582-9e9e-08c8e7ed16a0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:22.951331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d94797-ca95-4582-9e9e-08c8e7ed16a0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:22.951331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d94797-ca95-4582-9e9e-08c8e7ed16a0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:22.952763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 561/1139] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:17:22.957845 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda26416-c84a-4670-9b1c-c828a30a86fb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:22.958268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda26416-c84a-4670-9b1c-c828a30a86fb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:22.958410 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda26416-c84a-4670-9b1c-c828a30a86fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:22.975294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fda26416-c84a-4670-9b1c-c828a30a86fb None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:22.976476 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fda26416-c84a-4670-9b1c-c828a30a86fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:23.040634 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 579/1140] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:17:23.063648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d94797-ca95-4582-9e9e-08c8e7ed16a0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:23.160052 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d94797-ca95-4582-9e9e-08c8e7ed16a0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:23.161606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:23.195665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324428b9-f927-4e47-9250-2d6459744ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:23.196446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324428b9-f927-4e47-9250-2d6459744ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:23.196756 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324428b9-f927-4e47-9250-2d6459744ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:23.197018 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324428b9-f927-4e47-9250-2d6459744ff6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:23.206882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324428b9-f927-4e47-9250-2d6459744ff6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:17:23.217065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-324428b9-f927-4e47-9250-2d6459744ff6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:17:23.217065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:23.217065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:17:23.217065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:17:23.217065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:17:23.217065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:17:23.217065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:17:23.217065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:17:23.217065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:17:23.217065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:17:23.217065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:17:23.217065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:17:23.217065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:17:23.217065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:17:23.217065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:17:23.217065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:17:23.218347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:17:23.218347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:17:23.218347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:17:23.218347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:17:23.218347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:17:23.218347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:17:23.218347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:17:23.218347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:17:23.218347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:17:23.218347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:17:23.218347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:17:23.218347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:17:23.218347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:17:23.218347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:17:23.218347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:17:23.218347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:17:23.219638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:17:23.219638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:17:23.219638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:17:23.219638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:17:23.219638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:17:23.219638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:17:23.219638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:17:23.219638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:17:23.219638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:17:23.219638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:17:23.219638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:17:23.219638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:17:23.219638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 562/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:17:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:17:23.225239 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda26416-c84a-4670-9b1c-c828a30a86fb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:23.308572 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:23.333310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda26416-c84a-4670-9b1c-c828a30a86fb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:23.366919 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398fa03-4762-4319-8ef5-a1f2b7d5b014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:23.367575 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398fa03-4762-4319-8ef5-a1f2b7d5b014 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:23.367858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398fa03-4762-4319-8ef5-a1f2b7d5b014 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:23.368134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398fa03-4762-4319-8ef5-a1f2b7d5b014 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:23.368915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398fa03-4762-4319-8ef5-a1f2b7d5b014 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:23.369226 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398fa03-4762-4319-8ef5-a1f2b7d5b014 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:23.370595 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398fa03-4762-4319-8ef5-a1f2b7d5b014 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:23.378336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 580/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:17:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:23.384777 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324428b9-f927-4e47-9250-2d6459744ff6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:23.486146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324428b9-f927-4e47-9250-2d6459744ff6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:23.518682 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae2c71-fbcf-4995-8e5c-bffcff403e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:23.519545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae2c71-fbcf-4995-8e5c-bffcff403e1a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:23.519829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae2c71-fbcf-4995-8e5c-bffcff403e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:23.520104 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae2c71-fbcf-4995-8e5c-bffcff403e1a None admin] PATH_INFO: `/v3/users/84490e9a960947abab1f7314c5c281a5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:23.532913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae2c71-fbcf-4995-8e5c-bffcff403e1a None admin] RBAC: Authorizing `identity:get_user(user_id=84490e9a960947abab1f7314c5c281a5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:23.534342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae2c71-fbcf-4995-8e5c-bffcff403e1a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:23.536000 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 563/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:17:23 2022] GET /identity/v3/users/84490e9a960947abab1f7314c5c281a5 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:23.541876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398fa03-4762-4319-8ef5-a1f2b7d5b014 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:23.639045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398fa03-4762-4319-8ef5-a1f2b7d5b014 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:23.671691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941c799-2c35-4129-ac16-0c241bb60f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:23.672327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941c799-2c35-4129-ac16-0c241bb60f56 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:23.672635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941c799-2c35-4129-ac16-0c241bb60f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:23.672914 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941c799-2c35-4129-ac16-0c241bb60f56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:23.681939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:23.743275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941c799-2c35-4129-ac16-0c241bb60f56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:23.744689 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941c799-2c35-4129-ac16-0c241bb60f56 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:23.839404 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 581/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:17:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:17:23.845362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae2c71-fbcf-4995-8e5c-bffcff403e1a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:23.867487 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:23.941516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae2c71-fbcf-4995-8e5c-bffcff403e1a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:23.973304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9cb9d-d6f8-4e84-a1ca-cb38004e5ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:23.974148 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9cb9d-d6f8-4e84-a1ca-cb38004e5ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:23.975024 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9cb9d-d6f8-4e84-a1ca-cb38004e5ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:23.975304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9cb9d-d6f8-4e84-a1ca-cb38004e5ee4 None admin] PATH_INFO: `/v3/projects/2d7755cad7214c5f9d189d8f15f32c93` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:23.976478 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9cb9d-d6f8-4e84-a1ca-cb38004e5ee4 None admin] RBAC: Authorizing `identity:get_project(project_id=2d7755cad7214c5f9d189d8f15f32c93)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:23.977923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff9cb9d-d6f8-4e84-a1ca-cb38004e5ee4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:23.979599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 564/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:17:23 2022] GET /identity/v3/projects/2d7755cad7214c5f9d189d8f15f32c93 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:23.985920 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941c799-2c35-4129-ac16-0c241bb60f56 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:24.024343 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:24.081411 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941c799-2c35-4129-ac16-0c241bb60f56 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:24.114680 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b44e91-f918-4dc7-9a96-9e6f9fc25bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:24.115340 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b44e91-f918-4dc7-9a96-9e6f9fc25bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:24.115624 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b44e91-f918-4dc7-9a96-9e6f9fc25bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:24.115899 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b44e91-f918-4dc7-9a96-9e6f9fc25bc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:24.116748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b44e91-f918-4dc7-9a96-9e6f9fc25bc6 None admin] RBAC: Adding query filter params (role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.project.id=2d7755cad7214c5f9d189d8f15f32c93, user.id=84490e9a960947abab1f7314c5c281a5) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:24.117059 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b44e91-f918-4dc7-9a96-9e6f9fc25bc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:24.118480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b44e91-f918-4dc7-9a96-9e6f9fc25bc6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:24.123729 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 582/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:17:23 2022] GET /identity/v3/role_assignments?user.id=84490e9a960947abab1f7314c5c281a5&scope.project.id=2d7755cad7214c5f9d189d8f15f32c93&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:26.076799 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc0ff1-43ee-4094-8bb5-0710092e9697 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:26.077334 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc0ff1-43ee-4094-8bb5-0710092e9697 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:26.077334 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc0ff1-43ee-4094-8bb5-0710092e9697 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:26.078667 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 565/1147] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:17:26.083782 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d18c7-932f-44f3-9c89-d8cd9b8c613a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:26.084741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d18c7-932f-44f3-9c89-d8cd9b8c613a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:26.084958 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d18c7-932f-44f3-9c89-d8cd9b8c613a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:26.101322 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-422d18c7-932f-44f3-9c89-d8cd9b8c613a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:26.102505 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-422d18c7-932f-44f3-9c89-d8cd9b8c613a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:26.167572 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 583/1148] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:17:26.189481 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc0ff1-43ee-4094-8bb5-0710092e9697 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:26.275066 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:26.287859 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc0ff1-43ee-4094-8bb5-0710092e9697 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:26.320713 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3aa61-a0d4-4ae0-9e62-71d833e395f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:26.321477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3aa61-a0d4-4ae0-9e62-71d833e395f6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:26.321756 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3aa61-a0d4-4ae0-9e62-71d833e395f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:26.322024 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3aa61-a0d4-4ae0-9e62-71d833e395f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:26.331729 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3aa61-a0d4-4ae0-9e62-71d833e395f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:17:26.339195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e9a3aa61-a0d4-4ae0-9e62-71d833e395f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:17:26.339195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:26.339195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:17:26.339195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:17:26.339195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:17:26.339195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:17:26.339195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:17:26.339195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:17:26.339195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:17:26.339195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:17:26.339195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:17:26.339195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:17:26.339195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:17:26.339195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:17:26.339195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:17:26.339195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:17:26.340294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:17:26.340294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:17:26.340294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:17:26.340294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:17:26.340294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:17:26.340294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:17:26.340294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:17:26.340294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:17:26.340294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:17:26.340294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:17:26.340294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:17:26.340294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:17:26.340294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:17:26.340294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:17:26.340294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:17:26.340294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:17:26.341405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:17:26.341405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:17:26.341405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:17:26.341405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:17:26.341405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:17:26.341405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:17:26.341405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:17:26.341405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:17:26.341405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:17:26.341405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:17:26.341405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:17:26.341405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:17:26.341405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 566/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:17:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:17:26.348354 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d18c7-932f-44f3-9c89-d8cd9b8c613a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:26.378202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:26.444025 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d18c7-932f-44f3-9c89-d8cd9b8c613a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:26.476391 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee70ea-9a7f-4a0b-afcb-13206e99444e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:26.477068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee70ea-9a7f-4a0b-afcb-13206e99444e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:26.477354 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee70ea-9a7f-4a0b-afcb-13206e99444e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:26.477632 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee70ea-9a7f-4a0b-afcb-13206e99444e None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:26.478379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee70ea-9a7f-4a0b-afcb-13206e99444e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:26.478684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee70ea-9a7f-4a0b-afcb-13206e99444e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:26.480042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee70ea-9a7f-4a0b-afcb-13206e99444e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:26.488075 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 584/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:17:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:26.494311 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3aa61-a0d4-4ae0-9e62-71d833e395f6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:26.589077 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3aa61-a0d4-4ae0-9e62-71d833e395f6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:26.620954 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbb640-e45b-44f3-9a07-876a15463865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:26.621790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbb640-e45b-44f3-9a07-876a15463865 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:26.622069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbb640-e45b-44f3-9a07-876a15463865 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:26.622338 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbb640-e45b-44f3-9a07-876a15463865 None admin] PATH_INFO: `/v3/users/84490e9a960947abab1f7314c5c281a5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:26.623494 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbb640-e45b-44f3-9a07-876a15463865 None admin] RBAC: Authorizing `identity:get_user(user_id=84490e9a960947abab1f7314c5c281a5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:26.624940 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbb640-e45b-44f3-9a07-876a15463865 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:26.626526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 567/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:17:26 2022] GET /identity/v3/users/84490e9a960947abab1f7314c5c281a5 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:26.632700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee70ea-9a7f-4a0b-afcb-13206e99444e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:26.724808 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:26.732086 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee70ea-9a7f-4a0b-afcb-13206e99444e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:26.764425 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099f453-4893-4e1d-8dea-abd6e81fd767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:26.765294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099f453-4893-4e1d-8dea-abd6e81fd767 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:26.765579 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099f453-4893-4e1d-8dea-abd6e81fd767 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:26.765854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099f453-4893-4e1d-8dea-abd6e81fd767 None admin] PATH_INFO: `/v3/projects/2d7755cad7214c5f9d189d8f15f32c93` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:26.767022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099f453-4893-4e1d-8dea-abd6e81fd767 None admin] RBAC: Authorizing `identity:get_project(project_id=2d7755cad7214c5f9d189d8f15f32c93)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:26.768455 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099f453-4893-4e1d-8dea-abd6e81fd767 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:26.770175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 585/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:17:26 2022] GET /identity/v3/projects/2d7755cad7214c5f9d189d8f15f32c93 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:26.776255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbb640-e45b-44f3-9a07-876a15463865 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:26.794911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:26.874128 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbb640-e45b-44f3-9a07-876a15463865 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:26.905349 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10442222-2991-4950-9878-cb2aa62a74ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:26.906219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10442222-2991-4950-9878-cb2aa62a74ce None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:26.906518 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10442222-2991-4950-9878-cb2aa62a74ce None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:26.906776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10442222-2991-4950-9878-cb2aa62a74ce None admin] PATH_INFO: `/v3/projects/2d7755cad7214c5f9d189d8f15f32c93/users/84490e9a960947abab1f7314c5c281a5/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:26.908716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10442222-2991-4950-9878-cb2aa62a74ce None admin] RBAC: Authorizing `identity:create_grant(project_id=2d7755cad7214c5f9d189d8f15f32c93, user_id=84490e9a960947abab1f7314c5c281a5, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:26.910168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10442222-2991-4950-9878-cb2aa62a74ce None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:26.923264 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 568/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:17:26 2022] PUT /identity/v3/projects/2d7755cad7214c5f9d189d8f15f32c93/users/84490e9a960947abab1f7314c5c281a5/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:17:28.884758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc957f7e-64a9-4094-bb83-6bcb047d4a15 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:28.885362 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc957f7e-64a9-4094-bb83-6bcb047d4a15 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:28.885362 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc957f7e-64a9-4094-bb83-6bcb047d4a15 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:28.886698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 586/1154] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:17:28.891889 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b795fea-401e-4d2d-a682-5315dd818e3b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:28.892302 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b795fea-401e-4d2d-a682-5315dd818e3b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:28.892448 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b795fea-401e-4d2d-a682-5315dd818e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:28.909334 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5b795fea-401e-4d2d-a682-5315dd818e3b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:28.910513 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5b795fea-401e-4d2d-a682-5315dd818e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:28.975366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 569/1155] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:17:28.998882 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc957f7e-64a9-4094-bb83-6bcb047d4a15 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:29.097875 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc957f7e-64a9-4094-bb83-6bcb047d4a15 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:29.131191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97983c21-86d8-4a44-aa80-1abdff414f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:29.131961 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97983c21-86d8-4a44-aa80-1abdff414f1a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:29.132242 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97983c21-86d8-4a44-aa80-1abdff414f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:29.132536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97983c21-86d8-4a44-aa80-1abdff414f1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:29.142370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97983c21-86d8-4a44-aa80-1abdff414f1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:17:29.150502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-97983c21-86d8-4a44-aa80-1abdff414f1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:17:29.150502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:29.150502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:17:29.150502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:17:29.150502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:17:29.150502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:17:29.150502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:17:29.150502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:17:29.150502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:17:29.150502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:17:29.150502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:17:29.150502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:17:29.150502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:17:29.150502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:17:29.150502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:17:29.150502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:17:29.151480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:17:29.151480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:17:29.151480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:17:29.151480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:17:29.151480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:17:29.151480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:17:29.151480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:17:29.151480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:17:29.151480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:17:29.151480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:17:29.151480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:17:29.151480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:17:29.151480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:17:29.151480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:17:29.151480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:17:29.151480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:17:29.152336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:17:29.152336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:17:29.152336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:17:29.152336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:17:29.152336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:17:29.152336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:17:29.152336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:17:29.152336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:17:29.152336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:17:29.152336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:17:29.152336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:17:29.152336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:17:29.152336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 587/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:17:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:17:29.157759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b795fea-401e-4d2d-a682-5315dd818e3b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:29.166604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:29.257813 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b795fea-401e-4d2d-a682-5315dd818e3b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:29.291513 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e45595-0c23-4f96-95c1-871d1d1bb7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:29.291513 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e45595-0c23-4f96-95c1-871d1d1bb7ea None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:29.291513 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e45595-0c23-4f96-95c1-871d1d1bb7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:29.291513 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e45595-0c23-4f96-95c1-871d1d1bb7ea None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:29.292759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e45595-0c23-4f96-95c1-871d1d1bb7ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:29.293981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e45595-0c23-4f96-95c1-871d1d1bb7ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:29.295029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e45595-0c23-4f96-95c1-871d1d1bb7ea None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:29.310923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 570/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:17:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:29.317105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97983c21-86d8-4a44-aa80-1abdff414f1a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:29.342472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:29.414894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97983c21-86d8-4a44-aa80-1abdff414f1a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:29.446787 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726cf292-2d3e-46cf-b491-8855ed2f151e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:29.447635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cf292-2d3e-46cf-b491-8855ed2f151e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:29.447917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cf292-2d3e-46cf-b491-8855ed2f151e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:29.448195 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cf292-2d3e-46cf-b491-8855ed2f151e None admin] PATH_INFO: `/v3/users/84490e9a960947abab1f7314c5c281a5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:29.449390 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726cf292-2d3e-46cf-b491-8855ed2f151e None admin] RBAC: Authorizing `identity:get_user(user_id=84490e9a960947abab1f7314c5c281a5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:29.450811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726cf292-2d3e-46cf-b491-8855ed2f151e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:29.452403 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 588/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:17:29 2022] GET /identity/v3/users/84490e9a960947abab1f7314c5c281a5 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:29.458264 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e45595-0c23-4f96-95c1-871d1d1bb7ea None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:29.521806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:29.553539 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e45595-0c23-4f96-95c1-871d1d1bb7ea None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:29.585271 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2fb420-160e-4956-b3f7-4f9bd0667a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:29.585902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fb420-160e-4956-b3f7-4f9bd0667a64 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:29.586184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fb420-160e-4956-b3f7-4f9bd0667a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:29.586457 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fb420-160e-4956-b3f7-4f9bd0667a64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:29.649736 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2fb420-160e-4956-b3f7-4f9bd0667a64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:29.651160 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2fb420-160e-4956-b3f7-4f9bd0667a64 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:29.711689 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:29.745836 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 571/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:17:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:17:29.751703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726cf292-2d3e-46cf-b491-8855ed2f151e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:29.838379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:29.847996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726cf292-2d3e-46cf-b491-8855ed2f151e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:29.880000 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977540a6-2325-466b-8597-f783514f1ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:29.881370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977540a6-2325-466b-8597-f783514f1ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:29.881653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977540a6-2325-466b-8597-f783514f1ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:29.881930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977540a6-2325-466b-8597-f783514f1ac1 None admin] PATH_INFO: `/v3/projects/2d7755cad7214c5f9d189d8f15f32c93` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:29.883099 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977540a6-2325-466b-8597-f783514f1ac1 None admin] RBAC: Authorizing `identity:get_project(project_id=2d7755cad7214c5f9d189d8f15f32c93)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:29.884543 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977540a6-2325-466b-8597-f783514f1ac1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:29.886233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 589/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:17:29 2022] GET /identity/v3/projects/2d7755cad7214c5f9d189d8f15f32c93 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:29.892266 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2fb420-160e-4956-b3f7-4f9bd0667a64 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:29.988116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2fb420-160e-4956-b3f7-4f9bd0667a64 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:30.020364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd2b0c-6abd-4ff9-8210-b98605651080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:30.021028 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd2b0c-6abd-4ff9-8210-b98605651080 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:30.021856 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd2b0c-6abd-4ff9-8210-b98605651080 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:30.022136 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd2b0c-6abd-4ff9-8210-b98605651080 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:30.022959 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd2b0c-6abd-4ff9-8210-b98605651080 None admin] RBAC: Adding query filter params (role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.project.id=2d7755cad7214c5f9d189d8f15f32c93, user.id=84490e9a960947abab1f7314c5c281a5) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:30.023268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd2b0c-6abd-4ff9-8210-b98605651080 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:30.025104 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd2b0c-6abd-4ff9-8210-b98605651080 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:30.030730 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 572/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:17:29 2022] GET /identity/v3/role_assignments?user.id=84490e9a960947abab1f7314c5c281a5&scope.project.id=2d7755cad7214c5f9d189d8f15f32c93&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:17:32.039554 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec0c57-a8fd-4d04-83dd-17e4e96eb4fc None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:32.040083 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec0c57-a8fd-4d04-83dd-17e4e96eb4fc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:32.040177 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec0c57-a8fd-4d04-83dd-17e4e96eb4fc None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:32.041523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 590/1162] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:17:32.046722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcc043-3f8b-4d4c-9f30-d6c21031bfc9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:32.047109 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcc043-3f8b-4d4c-9f30-d6c21031bfc9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:32.047287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcc043-3f8b-4d4c-9f30-d6c21031bfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:32.064122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-30fcc043-3f8b-4d4c-9f30-d6c21031bfc9 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:32.065329 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-30fcc043-3f8b-4d4c-9f30-d6c21031bfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:32.090763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:32.129658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 573/1163] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:17:32.152465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec0c57-a8fd-4d04-83dd-17e4e96eb4fc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:32.252623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec0c57-a8fd-4d04-83dd-17e4e96eb4fc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:32.285462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209ec55-6b8a-4dee-9319-5a6f7202d649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:32.286104 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209ec55-6b8a-4dee-9319-5a6f7202d649 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:32.286394 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209ec55-6b8a-4dee-9319-5a6f7202d649 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:32.286674 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209ec55-6b8a-4dee-9319-5a6f7202d649 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:32.312697 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:32.353044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0209ec55-6b8a-4dee-9319-5a6f7202d649 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:32.354493 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0209ec55-6b8a-4dee-9319-5a6f7202d649 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:32.448313 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 591/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:17:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:17:32.454326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcc043-3f8b-4d4c-9f30-d6c21031bfc9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:32.549351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcc043-3f8b-4d4c-9f30-d6c21031bfc9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:32.581070 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95e64c-526b-4dbb-8d4f-8c96e2c0375e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:32.581798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95e64c-526b-4dbb-8d4f-8c96e2c0375e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:32.582084 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95e64c-526b-4dbb-8d4f-8c96e2c0375e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:32.582359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95e64c-526b-4dbb-8d4f-8c96e2c0375e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:32.589394 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95e64c-526b-4dbb-8d4f-8c96e2c0375e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:32.590823 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95e64c-526b-4dbb-8d4f-8c96e2c0375e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:32.605113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ce95e64c-526b-4dbb-8d4f-8c96e2c0375e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 10:17:32.605113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:32.605113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 18 10:17:32.605113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 18 10:17:32.605113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:17:32.605113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:17:32.605113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:17:32.605113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:17:32.605113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 18 10:17:32.605113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:17:32.605113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:17:32.605113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:17:32.605113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:32.605113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:17:32.605113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:17:32.605113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:17:32.605113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:17:32.606665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:17:32.606665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:17:32.606665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:17:32.606665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:17:32.606665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:17:32.606665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:17:32.606665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:17:32.606665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:17:32.606665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:17:32.606665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:17:32.606665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:17:32.606665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:17:32.606665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:17:32.606665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:17:32.606665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:17:32.606665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:17:32.607820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:17:32.607820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:17:32.607820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:17:32.607820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:17:32.607820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:17:32.607820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:17:32.607820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:17:32.607820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:17:32.607820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:17:32.607820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:17:32.607820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 18 10:17:32.607820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:17:32.607820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 10:17:32.607820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:17:32.607820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 574/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 18 10:17:32 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:17:32.612785 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209ec55-6b8a-4dee-9319-5a6f7202d649 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:32.672711 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:32.712885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209ec55-6b8a-4dee-9319-5a6f7202d649 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:32.746908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14aee29-300f-4e22-b3fe-3e112746683e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:32.747555 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14aee29-300f-4e22-b3fe-3e112746683e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:32.747836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14aee29-300f-4e22-b3fe-3e112746683e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:32.748107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14aee29-300f-4e22-b3fe-3e112746683e None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:32.748854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14aee29-300f-4e22-b3fe-3e112746683e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:32.749156 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14aee29-300f-4e22-b3fe-3e112746683e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:32.750504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14aee29-300f-4e22-b3fe-3e112746683e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:32.757383 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 592/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jul 18 10:17:32 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:34.714163 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6094fa2-c718-48ec-b433-a6e188a151e1 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:34.714682 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6094fa2-c718-48ec-b433-a6e188a151e1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:34.714800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6094fa2-c718-48ec-b433-a6e188a151e1 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:34.716100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 575/1167] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:17:34.721368 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dcb10-3e9c-4999-8266-49b380957518 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:34.721783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dcb10-3e9c-4999-8266-49b380957518 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:34.721933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dcb10-3e9c-4999-8266-49b380957518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:34.739020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e25dcb10-3e9c-4999-8266-49b380957518 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:34.740328 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e25dcb10-3e9c-4999-8266-49b380957518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:34.803875 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 593/1168] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:17:34.826108 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6094fa2-c718-48ec-b433-a6e188a151e1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:34.851364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:34.926236 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6094fa2-c718-48ec-b433-a6e188a151e1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:34.959165 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d594fc4-e25e-4382-850f-832c88ea3f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:34.959809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d594fc4-e25e-4382-850f-832c88ea3f42 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:34.960093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d594fc4-e25e-4382-850f-832c88ea3f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:34.960367 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d594fc4-e25e-4382-850f-832c88ea3f42 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:34.961128 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d594fc4-e25e-4382-850f-832c88ea3f42 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:34.962489 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d594fc4-e25e-4382-850f-832c88ea3f42 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:34.976280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 576/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Mon Jul 18 10:17:34 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:17:37.002859 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de725c3f-c1e8-44c6-8546-bc847d1362b9 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:37.003418 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de725c3f-c1e8-44c6-8546-bc847d1362b9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:37.003418 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de725c3f-c1e8-44c6-8546-bc847d1362b9 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:37.004866 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 594/1170] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:17:37.010513 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d0d00-606a-4793-87fa-f65e50e9ad9f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:37.010882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d0d00-606a-4793-87fa-f65e50e9ad9f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:37.011084 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d0d00-606a-4793-87fa-f65e50e9ad9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:37.027312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5f0d0d00-606a-4793-87fa-f65e50e9ad9f None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:37.028490 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5f0d0d00-606a-4793-87fa-f65e50e9ad9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:37.092291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 577/1171] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:17:37.115877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de725c3f-c1e8-44c6-8546-bc847d1362b9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:37.168707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:37.215104 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de725c3f-c1e8-44c6-8546-bc847d1362b9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:37.248266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b4be3-fd3f-4ffc-9f64-8a42bd99a8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:37.249130 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b4be3-fd3f-4ffc-9f64-8a42bd99a8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:37.249408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b4be3-fd3f-4ffc-9f64-8a42bd99a8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:37.249676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b4be3-fd3f-4ffc-9f64-8a42bd99a8c9 None admin] PATH_INFO: `/v3/domains/39a46cc07bc74b55ab0230bb6d0b9790` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:37.260184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b4be3-fd3f-4ffc-9f64-8a42bd99a8c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=39a46cc07bc74b55ab0230bb6d0b9790)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:37.261584 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b4be3-fd3f-4ffc-9f64-8a42bd99a8c9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:37.263244 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 595/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Mon Jul 18 10:17:37 2022] GET /identity/v3/domains/39a46cc07bc74b55ab0230bb6d0b9790 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:37.269130 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d0d00-606a-4793-87fa-f65e50e9ad9f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:37.283051 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:37.365619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d0d00-606a-4793-87fa-f65e50e9ad9f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:37.402908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085ed8d-ff95-4d32-bd65-bdb56d6ff2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:37.403572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085ed8d-ff95-4d32-bd65-bdb56d6ff2a3 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:37.403852 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085ed8d-ff95-4d32-bd65-bdb56d6ff2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:37.404124 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085ed8d-ff95-4d32-bd65-bdb56d6ff2a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:37.404872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9085ed8d-ff95-4d32-bd65-bdb56d6ff2a3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:37.406295 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9085ed8d-ff95-4d32-bd65-bdb56d6ff2a3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:37.429984 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 578/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 18 10:17:37 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:17:39.449988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a927a-d13f-4a49-b941-ad8052655084 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:39.450586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a927a-d13f-4a49-b941-ad8052655084 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:39.450586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a927a-d13f-4a49-b941-ad8052655084 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:39.451829 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 596/1174] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:17:39.457120 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc2e66-0f15-4728-9bf5-9a7913123c77 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:39.457628 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc2e66-0f15-4728-9bf5-9a7913123c77 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:39.457730 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc2e66-0f15-4728-9bf5-9a7913123c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:39.474711 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ebdc2e66-0f15-4728-9bf5-9a7913123c77 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:39.475894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ebdc2e66-0f15-4728-9bf5-9a7913123c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:39.533203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:39.542106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 579/1175] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:17:39.566425 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a927a-d13f-4a49-b941-ad8052655084 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:39.666103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a927a-d13f-4a49-b941-ad8052655084 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:39.678641 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:39.699436 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b85d7-e3a8-468c-b457-c594e02bb7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:39.700336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b85d7-e3a8-468c-b457-c594e02bb7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:39.700601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b85d7-e3a8-468c-b457-c594e02bb7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:39.700884 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b85d7-e3a8-468c-b457-c594e02bb7d1 None admin] PATH_INFO: `/v3/domains/39a46cc07bc74b55ab0230bb6d0b9790` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:39.702055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b85d7-e3a8-468c-b457-c594e02bb7d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=39a46cc07bc74b55ab0230bb6d0b9790)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:39.703487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b85d7-e3a8-468c-b457-c594e02bb7d1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:39.705125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 597/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Mon Jul 18 10:17:39 2022] GET /identity/v3/domains/39a46cc07bc74b55ab0230bb6d0b9790 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:39.711391 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc2e66-0f15-4728-9bf5-9a7913123c77 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:39.804796 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc2e66-0f15-4728-9bf5-9a7913123c77 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:39.836892 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ab708-8f73-4786-83bf-4c1e9e6abab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:39.837542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ab708-8f73-4786-83bf-4c1e9e6abab4 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:39.837826 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ab708-8f73-4786-83bf-4c1e9e6abab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:39.838102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ab708-8f73-4786-83bf-4c1e9e6abab4 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:39.838809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ab708-8f73-4786-83bf-4c1e9e6abab4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:39.840276 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ab708-8f73-4786-83bf-4c1e9e6abab4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:39.859039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 580/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 18 10:17:39 2022] POST /identity/v3/users => generated 311 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:17:42.014156 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4b773-04e5-46fa-85c1-f669ebef89b0 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:42.014716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4b773-04e5-46fa-85c1-f669ebef89b0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:42.014716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4b773-04e5-46fa-85c1-f669ebef89b0 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:42.016140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 598/1178] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:17:42.021976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b69ce-d119-449e-9331-2eee2233207b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:42.022407 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b69ce-d119-449e-9331-2eee2233207b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:42.022535 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b69ce-d119-449e-9331-2eee2233207b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:42.038718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a55b69ce-d119-449e-9331-2eee2233207b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:42.039884 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a55b69ce-d119-449e-9331-2eee2233207b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:42.084293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:42.104672 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 581/1179] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:17:42.128134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4b773-04e5-46fa-85c1-f669ebef89b0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:42.226330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4b773-04e5-46fa-85c1-f669ebef89b0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:42.259096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38b1d5-26e4-4c4e-b990-2d14417b8e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:42.259863 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38b1d5-26e4-4c4e-b990-2d14417b8e76 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:42.260139 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38b1d5-26e4-4c4e-b990-2d14417b8e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:42.260407 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38b1d5-26e4-4c4e-b990-2d14417b8e76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:42.270154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38b1d5-26e4-4c4e-b990-2d14417b8e76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:17:42.277679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-2c38b1d5-26e4-4c4e-b990-2d14417b8e76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:17:42.277679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:42.277679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:17:42.277679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:17:42.277679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:17:42.277679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:17:42.277679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:17:42.277679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:17:42.277679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:17:42.277679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:17:42.277679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:17:42.277679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:17:42.277679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:17:42.277679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:17:42.277679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:17:42.277679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:17:42.278684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:17:42.278684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:17:42.278684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:17:42.278684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:17:42.278684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:17:42.278684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:17:42.278684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:17:42.278684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:17:42.278684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:17:42.278684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:17:42.278684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:17:42.278684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:17:42.278684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:17:42.278684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:17:42.278684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:17:42.278684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:17:42.279710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:17:42.279710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:17:42.279710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:17:42.279710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:17:42.279710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:17:42.279710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:17:42.279710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:17:42.279710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:17:42.279710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:17:42.279710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:17:42.279710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:17:42.279710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:17:42.279710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 599/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:17:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:17:42.285093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b69ce-d119-449e-9331-2eee2233207b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:42.379368 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b69ce-d119-449e-9331-2eee2233207b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:42.412142 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0453d-56f9-4f33-8574-05b68c240d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:42.412825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0453d-56f9-4f33-8574-05b68c240d2e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:42.413113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0453d-56f9-4f33-8574-05b68c240d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:42.413387 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0453d-56f9-4f33-8574-05b68c240d2e None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:42.414123 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0453d-56f9-4f33-8574-05b68c240d2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:42.414429 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0453d-56f9-4f33-8574-05b68c240d2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:42.415793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0453d-56f9-4f33-8574-05b68c240d2e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:42.423536 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 582/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:17:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:42.429662 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38b1d5-26e4-4c4e-b990-2d14417b8e76 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:42.469310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:42.524920 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38b1d5-26e4-4c4e-b990-2d14417b8e76 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:42.557343 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617a793-270f-4a03-96ac-1287d61b14a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:42.558188 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617a793-270f-4a03-96ac-1287d61b14a1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:42.558472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617a793-270f-4a03-96ac-1287d61b14a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:42.558747 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617a793-270f-4a03-96ac-1287d61b14a1 None admin] PATH_INFO: `/v3/users/aac35b2b3ada494dbffc88af04323f31` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:42.571930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617a793-270f-4a03-96ac-1287d61b14a1 None admin] RBAC: Authorizing `identity:get_user(user_id=aac35b2b3ada494dbffc88af04323f31)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:42.573387 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617a793-270f-4a03-96ac-1287d61b14a1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:42.575018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 600/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:17:42 2022] GET /identity/v3/users/aac35b2b3ada494dbffc88af04323f31 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:42.580788 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0453d-56f9-4f33-8574-05b68c240d2e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:42.623515 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:42.678589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0453d-56f9-4f33-8574-05b68c240d2e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:42.709826 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b554ab4-ea10-49a0-b2fe-dc4b420dcb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:42.710457 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b554ab4-ea10-49a0-b2fe-dc4b420dcb4a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:42.710742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b554ab4-ea10-49a0-b2fe-dc4b420dcb4a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:42.711021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b554ab4-ea10-49a0-b2fe-dc4b420dcb4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:42.775615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b554ab4-ea10-49a0-b2fe-dc4b420dcb4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:42.777027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b554ab4-ea10-49a0-b2fe-dc4b420dcb4a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:42.819086 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:42.873482 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 583/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:17:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:17:42.879273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617a793-270f-4a03-96ac-1287d61b14a1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:42.971355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:42.976018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617a793-270f-4a03-96ac-1287d61b14a1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:43.009070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf85031-319b-4d95-a502-bd23f446df79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:43.009916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf85031-319b-4d95-a502-bd23f446df79 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:43.010202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf85031-319b-4d95-a502-bd23f446df79 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:43.010481 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf85031-319b-4d95-a502-bd23f446df79 None admin] PATH_INFO: `/v3/projects/ba861ca143c245f7b2aaa3f497b3f725` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:43.011653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf85031-319b-4d95-a502-bd23f446df79 None admin] RBAC: Authorizing `identity:get_project(project_id=ba861ca143c245f7b2aaa3f497b3f725)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:43.013119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf85031-319b-4d95-a502-bd23f446df79 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:43.014824 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 601/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:17:42 2022] GET /identity/v3/projects/ba861ca143c245f7b2aaa3f497b3f725 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:43.021536 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b554ab4-ea10-49a0-b2fe-dc4b420dcb4a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:43.116765 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b554ab4-ea10-49a0-b2fe-dc4b420dcb4a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:43.149621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0f06b-d505-44e0-829c-4c793e896f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:43.150295 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0f06b-d505-44e0-829c-4c793e896f8b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:43.150580 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0f06b-d505-44e0-829c-4c793e896f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:43.150853 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0f06b-d505-44e0-829c-4c793e896f8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:43.151683 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0f06b-d505-44e0-829c-4c793e896f8b None admin] RBAC: Adding query filter params (role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.project.id=ba861ca143c245f7b2aaa3f497b3f725, user.id=aac35b2b3ada494dbffc88af04323f31) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:43.151995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0f06b-d505-44e0-829c-4c793e896f8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:43.153458 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0f06b-d505-44e0-829c-4c793e896f8b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:43.158901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 584/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:17:43 2022] GET /identity/v3/role_assignments?user.id=aac35b2b3ada494dbffc88af04323f31&scope.project.id=ba861ca143c245f7b2aaa3f497b3f725&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:45.133300 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d11036-c1ae-4919-ba3d-57690028c519 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:45.134369 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d11036-c1ae-4919-ba3d-57690028c519 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:45.134880 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d11036-c1ae-4919-ba3d-57690028c519 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:45.136482 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 602/1186] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:17:45.141896 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085ed10-0bed-4036-853c-f61329512878 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:45.142668 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085ed10-0bed-4036-853c-f61329512878 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:45.142950 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085ed10-0bed-4036-853c-f61329512878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:45.155543 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:45.160917 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f085ed10-0bed-4036-853c-f61329512878 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:45.162794 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f085ed10-0bed-4036-853c-f61329512878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:45.231904 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 585/1187] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:17:45.254238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d11036-c1ae-4919-ba3d-57690028c519 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:45.355234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d11036-c1ae-4919-ba3d-57690028c519 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:45.387559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0807d-ba0f-41bc-869b-e1ea92d6372c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:45.388274 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0807d-ba0f-41bc-869b-e1ea92d6372c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:45.388581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0807d-ba0f-41bc-869b-e1ea92d6372c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:45.388857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0807d-ba0f-41bc-869b-e1ea92d6372c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:45.399155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0807d-ba0f-41bc-869b-e1ea92d6372c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:17:45.406737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-5af0807d-ba0f-41bc-869b-e1ea92d6372c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:17:45.406737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:45.406737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:17:45.406737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:17:45.406737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:17:45.406737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:17:45.406737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:17:45.406737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:17:45.406737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:17:45.406737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:17:45.406737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:17:45.406737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:17:45.406737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:17:45.406737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:17:45.406737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:17:45.406737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:17:45.407858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:17:45.407858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:17:45.407858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:17:45.407858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:17:45.407858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:17:45.407858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:17:45.407858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:17:45.407858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:17:45.407858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:17:45.407858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:17:45.407858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:17:45.407858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:17:45.407858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:17:45.407858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:17:45.407858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:17:45.407858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:17:45.408938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:17:45.408938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:17:45.408938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:17:45.408938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:17:45.408938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:17:45.408938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:17:45.408938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:17:45.408938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:17:45.408938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:17:45.408938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:17:45.408938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:17:45.408938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:17:45.408938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 603/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:17:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:17:45.414005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085ed10-0bed-4036-853c-f61329512878 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:45.514880 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085ed10-0bed-4036-853c-f61329512878 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:45.546895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da75b0-626a-4ab3-95c1-04cb3e7762bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:45.547548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da75b0-626a-4ab3-95c1-04cb3e7762bf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:45.547830 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da75b0-626a-4ab3-95c1-04cb3e7762bf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:45.548105 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da75b0-626a-4ab3-95c1-04cb3e7762bf None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:45.548869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da75b0-626a-4ab3-95c1-04cb3e7762bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:45.549173 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da75b0-626a-4ab3-95c1-04cb3e7762bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:45.550526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da75b0-626a-4ab3-95c1-04cb3e7762bf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:45.552667 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:45.559022 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 586/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:17:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:45.564839 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0807d-ba0f-41bc-869b-e1ea92d6372c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:45.592842 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:45.663999 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0807d-ba0f-41bc-869b-e1ea92d6372c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:45.696043 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41be6b-29da-43d7-8e58-b9899f1c9f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:45.696916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41be6b-29da-43d7-8e58-b9899f1c9f72 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:45.697200 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41be6b-29da-43d7-8e58-b9899f1c9f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:45.697475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41be6b-29da-43d7-8e58-b9899f1c9f72 None admin] PATH_INFO: `/v3/users/aac35b2b3ada494dbffc88af04323f31` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:45.698647 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41be6b-29da-43d7-8e58-b9899f1c9f72 None admin] RBAC: Authorizing `identity:get_user(user_id=aac35b2b3ada494dbffc88af04323f31)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:45.700077 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41be6b-29da-43d7-8e58-b9899f1c9f72 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:45.701823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 604/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:17:45 2022] GET /identity/v3/users/aac35b2b3ada494dbffc88af04323f31 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:45.707493 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da75b0-626a-4ab3-95c1-04cb3e7762bf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:45.805022 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da75b0-626a-4ab3-95c1-04cb3e7762bf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:45.837350 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533c13c-0acc-4ec0-a8aa-7e1868749fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:45.838192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533c13c-0acc-4ec0-a8aa-7e1868749fec None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:45.838477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533c13c-0acc-4ec0-a8aa-7e1868749fec None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:45.838757 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533c13c-0acc-4ec0-a8aa-7e1868749fec None admin] PATH_INFO: `/v3/projects/ba861ca143c245f7b2aaa3f497b3f725` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:45.839917 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533c13c-0acc-4ec0-a8aa-7e1868749fec None admin] RBAC: Authorizing `identity:get_project(project_id=ba861ca143c245f7b2aaa3f497b3f725)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:45.841372 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533c13c-0acc-4ec0-a8aa-7e1868749fec None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:45.843059 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 587/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:17:45 2022] GET /identity/v3/projects/ba861ca143c245f7b2aaa3f497b3f725 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:45.849056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41be6b-29da-43d7-8e58-b9899f1c9f72 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:45.938979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:45.946550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41be6b-29da-43d7-8e58-b9899f1c9f72 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:45.978466 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25f9f0-f8b1-4c44-981d-a493454bec1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:45.979348 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25f9f0-f8b1-4c44-981d-a493454bec1f None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:45.979630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25f9f0-f8b1-4c44-981d-a493454bec1f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:45.979906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25f9f0-f8b1-4c44-981d-a493454bec1f None admin] PATH_INFO: `/v3/projects/ba861ca143c245f7b2aaa3f497b3f725/users/aac35b2b3ada494dbffc88af04323f31/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:45.982130 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25f9f0-f8b1-4c44-981d-a493454bec1f None admin] RBAC: Authorizing `identity:create_grant(project_id=ba861ca143c245f7b2aaa3f497b3f725, user_id=aac35b2b3ada494dbffc88af04323f31, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:45.983565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25f9f0-f8b1-4c44-981d-a493454bec1f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:45.996499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 605/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:17:45 2022] PUT /identity/v3/projects/ba861ca143c245f7b2aaa3f497b3f725/users/aac35b2b3ada494dbffc88af04323f31/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:17:47.954533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81cb8e-4fe8-4a1c-a231-19a62e6b877b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:47.955089 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81cb8e-4fe8-4a1c-a231-19a62e6b877b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:47.955089 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81cb8e-4fe8-4a1c-a231-19a62e6b877b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:47.956476 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 588/1193] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:17:47.962053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594f36f-482f-44bb-97ce-6945b66102a7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:47.962463 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594f36f-482f-44bb-97ce-6945b66102a7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:47.962610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594f36f-482f-44bb-97ce-6945b66102a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:47.978917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5594f36f-482f-44bb-97ce-6945b66102a7 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:17:47.980090 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5594f36f-482f-44bb-97ce-6945b66102a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:17:48.046763 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 606/1194] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:17:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:17:48.070769 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81cb8e-4fe8-4a1c-a231-19a62e6b877b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:48.134389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:48.169153 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81cb8e-4fe8-4a1c-a231-19a62e6b877b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:48.201991 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c5d61-61b0-462c-94cd-102841429213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:48.202765 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c5d61-61b0-462c-94cd-102841429213 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:48.203044 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c5d61-61b0-462c-94cd-102841429213 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:48.203319 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c5d61-61b0-462c-94cd-102841429213 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:48.213083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c5d61-61b0-462c-94cd-102841429213 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:17:48.220488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-751c5d61-61b0-462c-94cd-102841429213 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:17:48.220488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:17:48.220488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:17:48.220488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:17:48.220488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:17:48.220488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:17:48.220488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:17:48.220488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:17:48.220488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:17:48.220488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:17:48.220488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:17:48.220488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:17:48.220488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:17:48.220488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:17:48.220488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:17:48.220488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:17:48.221612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:17:48.221612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:17:48.221612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:17:48.221612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:17:48.221612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:17:48.221612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:17:48.221612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:17:48.221612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:17:48.221612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:17:48.221612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:17:48.221612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:17:48.221612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:17:48.221612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:17:48.221612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:17:48.221612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:17:48.221612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:17:48.222806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:17:48.222806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:17:48.222806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:17:48.222806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:17:48.222806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:17:48.222806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:17:48.222806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:17:48.222806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:17:48.222806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:17:48.222806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:17:48.222806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:17:48.222806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:17:48.222806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 589/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:17:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:17:48.227738 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594f36f-482f-44bb-97ce-6945b66102a7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:48.308726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:48.324866 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594f36f-482f-44bb-97ce-6945b66102a7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:48.358828 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae496fc7-8aea-4bab-8359-66ff17bb3279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:48.359481 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae496fc7-8aea-4bab-8359-66ff17bb3279 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:48.359767 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae496fc7-8aea-4bab-8359-66ff17bb3279 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:48.360044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae496fc7-8aea-4bab-8359-66ff17bb3279 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:48.360835 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae496fc7-8aea-4bab-8359-66ff17bb3279 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:48.361141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae496fc7-8aea-4bab-8359-66ff17bb3279 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:48.362506 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae496fc7-8aea-4bab-8359-66ff17bb3279 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:48.370333 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 607/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:17:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:48.376030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c5d61-61b0-462c-94cd-102841429213 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:48.472868 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c5d61-61b0-462c-94cd-102841429213 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:48.497127 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:48.506459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9185d8-9560-41ec-85f6-65053fbf4bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:48.507307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9185d8-9560-41ec-85f6-65053fbf4bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:48.507592 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9185d8-9560-41ec-85f6-65053fbf4bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:48.507870 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9185d8-9560-41ec-85f6-65053fbf4bd9 None admin] PATH_INFO: `/v3/users/aac35b2b3ada494dbffc88af04323f31` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:48.509064 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9185d8-9560-41ec-85f6-65053fbf4bd9 None admin] RBAC: Authorizing `identity:get_user(user_id=aac35b2b3ada494dbffc88af04323f31)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:48.510511 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9185d8-9560-41ec-85f6-65053fbf4bd9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:48.512116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 590/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jul 18 10:17:48 2022] GET /identity/v3/users/aac35b2b3ada494dbffc88af04323f31 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:48.517758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae496fc7-8aea-4bab-8359-66ff17bb3279 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:48.612285 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae496fc7-8aea-4bab-8359-66ff17bb3279 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:48.644123 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f06ab1-a395-4830-b309-fa096e42136d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:48.644776 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f06ab1-a395-4830-b309-fa096e42136d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:48.645058 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f06ab1-a395-4830-b309-fa096e42136d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:48.645332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f06ab1-a395-4830-b309-fa096e42136d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:48.659758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:48.710434 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f06ab1-a395-4830-b309-fa096e42136d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:48.711859 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f06ab1-a395-4830-b309-fa096e42136d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:48.806490 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 608/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:17:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:17:48.812311 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9185d8-9560-41ec-85f6-65053fbf4bd9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:48.907515 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9185d8-9560-41ec-85f6-65053fbf4bd9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:48.939465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c10907-bfd4-42bd-b8a1-618926229957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:48.940292 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c10907-bfd4-42bd-b8a1-618926229957 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:48.940598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c10907-bfd4-42bd-b8a1-618926229957 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:48.940875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c10907-bfd4-42bd-b8a1-618926229957 None admin] PATH_INFO: `/v3/projects/ba861ca143c245f7b2aaa3f497b3f725` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:48.942022 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c10907-bfd4-42bd-b8a1-618926229957 None admin] RBAC: Authorizing `identity:get_project(project_id=ba861ca143c245f7b2aaa3f497b3f725)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:48.943473 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c10907-bfd4-42bd-b8a1-618926229957 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:48.945153 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 591/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jul 18 10:17:48 2022] GET /identity/v3/projects/ba861ca143c245f7b2aaa3f497b3f725 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:48.951041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f06ab1-a395-4830-b309-fa096e42136d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:17:48.995822 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:49.046793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f06ab1-a395-4830-b309-fa096e42136d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:17:49.078618 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2fe376-7a3e-4cea-abe0-e0c8d2ea6263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:17:49.079198 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fe376-7a3e-4cea-abe0-e0c8d2ea6263 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:17:49.079482 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fe376-7a3e-4cea-abe0-e0c8d2ea6263 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:17:49.079758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fe376-7a3e-4cea-abe0-e0c8d2ea6263 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:17:49.080603 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2fe376-7a3e-4cea-abe0-e0c8d2ea6263 None admin] RBAC: Adding query filter params (role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.project.id=ba861ca143c245f7b2aaa3f497b3f725, user.id=aac35b2b3ada494dbffc88af04323f31) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:17:49.080916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2fe376-7a3e-4cea-abe0-e0c8d2ea6263 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:17:49.082333 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2fe376-7a3e-4cea-abe0-e0c8d2ea6263 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:17:49.088455 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 609/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:17:48 2022] GET /identity/v3/role_assignments?user.id=aac35b2b3ada494dbffc88af04323f31&scope.project.id=ba861ca143c245f7b2aaa3f497b3f725&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:17:58.935480 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=69 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:17:59.085892 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=45 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:43.815540 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7f5d9-51b8-489b-8455-e0eb0cb7f144 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:43.816124 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7f5d9-51b8-489b-8455-e0eb0cb7f144 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:43.816124 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7f5d9-51b8-489b-8455-e0eb0cb7f144 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:43.817723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 592/1201] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:18:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:18:43.826201 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010f766-e259-48e8-b4b5-2c2ebeb27c33 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:43.826661 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010f766-e259-48e8-b4b5-2c2ebeb27c33 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:43.826750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010f766-e259-48e8-b4b5-2c2ebeb27c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:43.849200 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1010f766-e259-48e8-b4b5-2c2ebeb27c33 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:18:43.850455 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1010f766-e259-48e8-b4b5-2c2ebeb27c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:18:43.942703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 610/1202] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:18:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:18:43.969973 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7f5d9-51b8-489b-8455-e0eb0cb7f144 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:44.093161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7f5d9-51b8-489b-8455-e0eb0cb7f144 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:44.149345 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836eb15-8ccd-4cd0-a71e-14ccc94cd997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:44.151265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836eb15-8ccd-4cd0-a71e-14ccc94cd997 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:44.151743 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836eb15-8ccd-4cd0-a71e-14ccc94cd997 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:44.152239 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836eb15-8ccd-4cd0-a71e-14ccc94cd997 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:44.164050 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836eb15-8ccd-4cd0-a71e-14ccc94cd997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:44.165567 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836eb15-8ccd-4cd0-a71e-14ccc94cd997 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:44.183732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c836eb15-8ccd-4cd0-a71e-14ccc94cd997 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:18:44.183732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:18:44.183732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:18:44.183732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:18:44.183732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:18:44.183732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:18:44.183732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:18:44.183732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:18:44.183732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:18:44.183732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:18:44.183732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:18:44.183732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:18:44.183732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:18:44.183732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:18:44.183732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:18:44.184869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:18:44.184869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:18:44.184869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:18:44.184869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:18:44.184869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:18:44.184869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:18:44.184869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:18:44.184869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:18:44.184869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:18:44.184869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:18:44.184869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:18:44.184869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:18:44.184869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:18:44.184869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:18:44.184869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:18:44.184869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:18:44.184869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:18:44.185922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:18:44.185922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:18:44.185922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:18:44.185922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:18:44.185922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 593/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:18:44.193663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1010f766-e259-48e8-b4b5-2c2ebeb27c33 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:44.314602 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1010f766-e259-48e8-b4b5-2c2ebeb27c33 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:44.349902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:44.355950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff687d-2459-49de-882c-a0d20463875d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:44.356914 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff687d-2459-49de-882c-a0d20463875d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:44.357408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff687d-2459-49de-882c-a0d20463875d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:44.357765 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff687d-2459-49de-882c-a0d20463875d None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:44.358625 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff687d-2459-49de-882c-a0d20463875d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:18:44.359035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff687d-2459-49de-882c-a0d20463875d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:44.361432 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff687d-2459-49de-882c-a0d20463875d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:44.378649 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 611/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:18:44.380746 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836eb15-8ccd-4cd0-a71e-14ccc94cd997 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:44.448341 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:44.512690 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836eb15-8ccd-4cd0-a71e-14ccc94cd997 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:44.555024 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05575be4-2a46-4f9d-8bd8-3c9eb097e42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:44.555771 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05575be4-2a46-4f9d-8bd8-3c9eb097e42b None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:44.556100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05575be4-2a46-4f9d-8bd8-3c9eb097e42b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:44.556419 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05575be4-2a46-4f9d-8bd8-3c9eb097e42b None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:44.557266 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05575be4-2a46-4f9d-8bd8-3c9eb097e42b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:44.558848 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05575be4-2a46-4f9d-8bd8-3c9eb097e42b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:44.583323 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 594/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jul 18 10:18:44 2022] POST /identity/v3/users => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:18:46.773309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517d415-a0c8-475d-9505-d1038e706ba2 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:46.773922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517d415-a0c8-475d-9505-d1038e706ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:46.773922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517d415-a0c8-475d-9505-d1038e706ba2 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:46.775510 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 612/1206] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:18:46.782363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ffe4d-832b-446d-9b41-0d69c1a35bb1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:46.782871 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ffe4d-832b-446d-9b41-0d69c1a35bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:46.782987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ffe4d-832b-446d-9b41-0d69c1a35bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:46.801854 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a62ffe4d-832b-446d-9b41-0d69c1a35bb1 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:18:46.803105 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a62ffe4d-832b-446d-9b41-0d69c1a35bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:18:46.877486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 595/1207] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:18:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:18:46.902158 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517d415-a0c8-475d-9505-d1038e706ba2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:47.030797 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517d415-a0c8-475d-9505-d1038e706ba2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:47.069119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818089d7-4395-49ff-ae44-7f03519bc9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:47.069928 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818089d7-4395-49ff-ae44-7f03519bc9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:47.070245 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818089d7-4395-49ff-ae44-7f03519bc9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:47.070542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818089d7-4395-49ff-ae44-7f03519bc9a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:47.089226 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818089d7-4395-49ff-ae44-7f03519bc9a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:18:47.097342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-818089d7-4395-49ff-ae44-7f03519bc9a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:18:47.097342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:18:47.097342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:18:47.097342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:18:47.097342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:18:47.097342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:18:47.097342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:18:47.097342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:18:47.097342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:18:47.097342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:18:47.097342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:18:47.097342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:18:47.097342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:18:47.097342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:18:47.097342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:18:47.097342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:18:47.098780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:18:47.098780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:18:47.098780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:18:47.098780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:18:47.098780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:18:47.098780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:18:47.098780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:18:47.098780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:18:47.098780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:18:47.098780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:18:47.098780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:18:47.098780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:18:47.098780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:18:47.098780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:18:47.098780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:18:47.098780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:18:47.099643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:18:47.099643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:18:47.099643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:18:47.099643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:18:47.099643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:18:47.099643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:18:47.099643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:18:47.099643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:18:47.099643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:18:47.099643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:18:47.099643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:18:47.099643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:18:47.099643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 613/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:18:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:18:47.104772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ffe4d-832b-446d-9b41-0d69c1a35bb1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:47.151084 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:47.225280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ffe4d-832b-446d-9b41-0d69c1a35bb1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:47.262860 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f590cb3-dce9-4989-b8ac-217040504014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:47.263599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f590cb3-dce9-4989-b8ac-217040504014 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:47.263884 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f590cb3-dce9-4989-b8ac-217040504014 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:47.264161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f590cb3-dce9-4989-b8ac-217040504014 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:47.264976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f590cb3-dce9-4989-b8ac-217040504014 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:18:47.265275 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f590cb3-dce9-4989-b8ac-217040504014 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:47.266704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f590cb3-dce9-4989-b8ac-217040504014 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:47.276409 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 596/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:18:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:18:47.282308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818089d7-4395-49ff-ae44-7f03519bc9a7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:47.341953 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:47.408183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818089d7-4395-49ff-ae44-7f03519bc9a7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:47.450161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ed6e6-11b5-4dbb-9483-5ba048d41fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:47.450997 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ed6e6-11b5-4dbb-9483-5ba048d41fca None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:47.451281 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ed6e6-11b5-4dbb-9483-5ba048d41fca None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:47.451597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ed6e6-11b5-4dbb-9483-5ba048d41fca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:47.463983 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ed6e6-11b5-4dbb-9483-5ba048d41fca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:47.465537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ed6e6-11b5-4dbb-9483-5ba048d41fca None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:47.484794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-061ed6e6-11b5-4dbb-9483-5ba048d41fca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:18:47.484794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:18:47.484794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:18:47.484794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:18:47.484794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:18:47.484794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:18:47.484794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:18:47.484794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:18:47.484794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:18:47.484794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:18:47.484794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:18:47.484794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:18:47.484794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:18:47.484794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:18:47.484794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:18:47.485851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:18:47.485851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:18:47.485851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:18:47.485851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:18:47.485851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:18:47.485851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:18:47.485851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:18:47.485851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:18:47.485851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:18:47.485851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:18:47.485851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:18:47.485851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:18:47.485851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:18:47.485851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:18:47.485851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:18:47.485851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:18:47.485851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:18:47.491883 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:18:47.491883 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:18:47.491883 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:18:47.491883 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:18:47.491883 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 614/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:18:47.492178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f590cb3-dce9-4989-b8ac-217040504014 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:47.609199 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:47.612636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f590cb3-dce9-4989-b8ac-217040504014 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:47.652448 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356e442-d1ec-4f21-b392-b4e4d3f37ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:47.653131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356e442-d1ec-4f21-b392-b4e4d3f37ced None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:47.654160 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356e442-d1ec-4f21-b392-b4e4d3f37ced None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:47.654160 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356e442-d1ec-4f21-b392-b4e4d3f37ced None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:47.654523 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356e442-d1ec-4f21-b392-b4e4d3f37ced None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:18:47.654836 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356e442-d1ec-4f21-b392-b4e4d3f37ced None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:47.656238 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356e442-d1ec-4f21-b392-b4e4d3f37ced None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:47.673063 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 597/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:18:47.674463 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ed6e6-11b5-4dbb-9483-5ba048d41fca None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:47.801492 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ed6e6-11b5-4dbb-9483-5ba048d41fca None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:47.804029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:47.840105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84118509-4ab6-42a2-ab30-299417c53847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:47.840944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84118509-4ab6-42a2-ab30-299417c53847 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:47.841231 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84118509-4ab6-42a2-ab30-299417c53847 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:47.841507 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84118509-4ab6-42a2-ab30-299417c53847 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:47.849928 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84118509-4ab6-42a2-ab30-299417c53847 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:47.851389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84118509-4ab6-42a2-ab30-299417c53847 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:47.866536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-84118509-4ab6-42a2-ab30-299417c53847 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:18:47.866536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:18:47.866536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:18:47.866536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:18:47.866536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:18:47.866536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:18:47.866536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:18:47.866536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:18:47.866536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:18:47.866536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:18:47.866536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:18:47.866536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:18:47.866536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:18:47.866536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:18:47.866536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:18:47.866536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:18:47.867848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:18:47.867848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:18:47.867848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:18:47.867848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:18:47.867848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:18:47.867848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:18:47.867848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:18:47.867848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:18:47.867848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:18:47.867848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:18:47.867848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:18:47.867848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:18:47.867848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:18:47.867848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:18:47.867848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:18:47.867848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:18:47.867848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:18:47.867848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:18:47.868905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:18:47.868905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:18:47.868905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:18:47.868905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:18:47.868905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:18:47.868905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:18:47.868905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:18:47.868905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:18:47.868905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:18:47.868905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:18:47.868905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 615/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:18:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:18:47.873610 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356e442-d1ec-4f21-b392-b4e4d3f37ced None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:47.993218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356e442-d1ec-4f21-b392-b4e4d3f37ced None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:48.032329 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffa61b-1700-4776-8515-bd7df12dc552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:48.033047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffa61b-1700-4776-8515-bd7df12dc552 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:48.033322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffa61b-1700-4776-8515-bd7df12dc552 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:48.033600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffa61b-1700-4776-8515-bd7df12dc552 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:48.034441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ffa61b-1700-4776-8515-bd7df12dc552 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:18:48.034749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ffa61b-1700-4776-8515-bd7df12dc552 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:48.036210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ffa61b-1700-4776-8515-bd7df12dc552 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:48.057844 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 598/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Jul 18 10:18:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:18:48.063809 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84118509-4ab6-42a2-ab30-299417c53847 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:48.186862 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84118509-4ab6-42a2-ab30-299417c53847 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:48.232913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c3c43-0225-4bfc-9ec5-06bbfb333bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:48.233581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c3c43-0225-4bfc-9ec5-06bbfb333bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:48.234050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c3c43-0225-4bfc-9ec5-06bbfb333bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:48.234335 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c3c43-0225-4bfc-9ec5-06bbfb333bc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:48.271285 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:48.314388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c3c43-0225-4bfc-9ec5-06bbfb333bc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:48.315875 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c3c43-0225-4bfc-9ec5-06bbfb333bc9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:48.432471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 616/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:18:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 10:18:48.439579 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffa61b-1700-4776-8515-bd7df12dc552 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:48.462424 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:48.559208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffa61b-1700-4776-8515-bd7df12dc552 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:48.607292 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfddc0-48dd-4427-9195-51663d8839ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:48.608219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfddc0-48dd-4427-9195-51663d8839ca None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:48.608599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfddc0-48dd-4427-9195-51663d8839ca None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:48.608820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfddc0-48dd-4427-9195-51663d8839ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:48.622167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdfddc0-48dd-4427-9195-51663d8839ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:48.622670 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdfddc0-48dd-4427-9195-51663d8839ca None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:48.646838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7fdfddc0-48dd-4427-9195-51663d8839ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:18:48.646838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:18:48.646838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:18:48.646838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:18:48.646838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:18:48.646838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:18:48.646838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:18:48.646838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:18:48.646838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:18:48.646838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:18:48.646838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:18:48.646838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:18:48.646838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:18:48.646838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:18:48.646838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:18:48.648564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:18:48.648564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:18:48.648564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:18:48.648564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:18:48.648564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:18:48.648564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:18:48.648564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:18:48.648564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:18:48.648564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:18:48.648564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:18:48.648564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:18:48.648564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:18:48.648564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:18:48.648564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:18:48.648564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:18:48.648564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:18:48.648564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:18:48.649602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:18:48.649602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:18:48.649602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:18:48.649602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:18:48.654381 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 599/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:18:48.655953 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c3c43-0225-4bfc-9ec5-06bbfb333bc9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:48.729852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:48.778340 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c3c43-0225-4bfc-9ec5-06bbfb333bc9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:48.822284 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1d420-4624-4c5c-b1f7-3e53f53f9ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:48.823073 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1d420-4624-4c5c-b1f7-3e53f53f9ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:48.823312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1d420-4624-4c5c-b1f7-3e53f53f9ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:48.823536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1d420-4624-4c5c-b1f7-3e53f53f9ea0 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:48.824319 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1d420-4624-4c5c-b1f7-3e53f53f9ea0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:18:48.824641 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1d420-4624-4c5c-b1f7-3e53f53f9ea0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:48.826337 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1d420-4624-4c5c-b1f7-3e53f53f9ea0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:48.839337 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 617/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:18:48.845648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfddc0-48dd-4427-9195-51663d8839ca None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:48.936422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:48.980121 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfddc0-48dd-4427-9195-51663d8839ca None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:49.019862 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124651c-a308-4a4f-8c92-bf7868be98ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:49.020684 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124651c-a308-4a4f-8c92-bf7868be98ab None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:49.020967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124651c-a308-4a4f-8c92-bf7868be98ab None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:49.021242 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124651c-a308-4a4f-8c92-bf7868be98ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:49.032689 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124651c-a308-4a4f-8c92-bf7868be98ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:49.034562 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124651c-a308-4a4f-8c92-bf7868be98ab None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:49.050221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8124651c-a308-4a4f-8c92-bf7868be98ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:18:49.050221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:18:49.050221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:18:49.050221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:18:49.050221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:18:49.050221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:18:49.050221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:18:49.050221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:18:49.050221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:18:49.050221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:18:49.050221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:18:49.050221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:18:49.050221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:18:49.050221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:18:49.050221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:18:49.051782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:18:49.051782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:18:49.051782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:18:49.051782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:18:49.051782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:18:49.051782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:18:49.051782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:18:49.051782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:18:49.051782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:18:49.051782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:18:49.051782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:18:49.051782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:18:49.051782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:18:49.051782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:18:49.051782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:18:49.051782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:18:49.051782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:18:49.053420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:18:49.053420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:18:49.053420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:18:49.053420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:18:49.053420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:18:49.053420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:18:49.053420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:18:49.053420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:18:49.053420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 600/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:18:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:18:49.058504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1d420-4624-4c5c-b1f7-3e53f53f9ea0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:49.180057 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1d420-4624-4c5c-b1f7-3e53f53f9ea0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:49.208308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:49.223838 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62318c0e-17ce-4eb7-ab9b-0cbe7fe7c161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:49.225039 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62318c0e-17ce-4eb7-ab9b-0cbe7fe7c161 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:49.225039 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62318c0e-17ce-4eb7-ab9b-0cbe7fe7c161 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:49.225214 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62318c0e-17ce-4eb7-ab9b-0cbe7fe7c161 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:49.228198 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62318c0e-17ce-4eb7-ab9b-0cbe7fe7c161 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:18:49.228198 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62318c0e-17ce-4eb7-ab9b-0cbe7fe7c161 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:49.228198 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62318c0e-17ce-4eb7-ab9b-0cbe7fe7c161 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:49.248627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 618/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:18:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:18:49.249709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124651c-a308-4a4f-8c92-bf7868be98ab None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:49.365725 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124651c-a308-4a4f-8c92-bf7868be98ab None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:49.400813 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:49.408027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcdccd-41a5-46da-9b3d-444dc9ba3a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:49.408959 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcdccd-41a5-46da-9b3d-444dc9ba3a68 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:49.409287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcdccd-41a5-46da-9b3d-444dc9ba3a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:49.409640 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcdccd-41a5-46da-9b3d-444dc9ba3a68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:49.411170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bcdccd-41a5-46da-9b3d-444dc9ba3a68 None admin] RBAC: Adding query filter params (role.id=f57b3e871ce142a090d6aa7dea9ac7d5, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=d07165f554a445169b2c3f0d173c0bb7) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:18:49.411486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bcdccd-41a5-46da-9b3d-444dc9ba3a68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:49.412993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bcdccd-41a5-46da-9b3d-444dc9ba3a68 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:49.420033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 601/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:18:49 2022] GET /identity/v3/role_assignments?user.id=d07165f554a445169b2c3f0d173c0bb7&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=f57b3e871ce142a090d6aa7dea9ac7d5 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:18:51.499111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a980dc8-9541-4c26-9461-79b5db2a9273 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:51.499654 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a980dc8-9541-4c26-9461-79b5db2a9273 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:51.499654 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a980dc8-9541-4c26-9461-79b5db2a9273 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:51.501100 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 619/1220] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:18:51.507936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcf287-44cc-496b-99b9-71ec63250737 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:51.508428 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcf287-44cc-496b-99b9-71ec63250737 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:51.508599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcf287-44cc-496b-99b9-71ec63250737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:51.529163 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-30dcf287-44cc-496b-99b9-71ec63250737 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:18:51.530947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-30dcf287-44cc-496b-99b9-71ec63250737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:18:51.611985 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 602/1221] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:18:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:18:51.635050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a980dc8-9541-4c26-9461-79b5db2a9273 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:51.749627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a980dc8-9541-4c26-9461-79b5db2a9273 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:51.785887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8becc22b-fa4f-47ce-818a-c77f5548e871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:51.787194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becc22b-fa4f-47ce-818a-c77f5548e871 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:51.787523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becc22b-fa4f-47ce-818a-c77f5548e871 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:51.787911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becc22b-fa4f-47ce-818a-c77f5548e871 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:51.801803 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8becc22b-fa4f-47ce-818a-c77f5548e871 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:18:51.809581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8becc22b-fa4f-47ce-818a-c77f5548e871 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:18:51.809581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:18:51.809581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:18:51.809581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:18:51.809581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:18:51.809581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:18:51.809581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:18:51.809581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:18:51.809581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:18:51.809581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:18:51.809581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:18:51.809581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:18:51.809581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:18:51.809581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:18:51.809581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:18:51.809581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:18:51.810813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:18:51.810813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:18:51.810813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:18:51.810813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:18:51.810813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:18:51.810813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:18:51.810813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:18:51.810813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:18:51.810813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:18:51.810813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:18:51.810813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:18:51.810813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:18:51.810813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:18:51.810813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:18:51.810813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:18:51.810813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:18:51.811664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:18:51.811664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:18:51.811664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:18:51.811664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:18:51.811664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:18:51.811664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:18:51.811664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:18:51.811664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:18:51.811664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:18:51.811664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:18:51.811664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:18:51.811664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:18:51.811664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 620/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:18:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:18:51.819864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcf287-44cc-496b-99b9-71ec63250737 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:51.930528 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcf287-44cc-496b-99b9-71ec63250737 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:51.956241 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:51.968169 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05862fa-be44-4e80-a240-8e069960606b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:51.968886 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05862fa-be44-4e80-a240-8e069960606b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:51.969157 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05862fa-be44-4e80-a240-8e069960606b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:51.969533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05862fa-be44-4e80-a240-8e069960606b None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:51.970268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05862fa-be44-4e80-a240-8e069960606b None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:18:51.970575 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05862fa-be44-4e80-a240-8e069960606b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:51.971944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05862fa-be44-4e80-a240-8e069960606b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:51.981955 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 603/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:18:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:18:51.987950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8becc22b-fa4f-47ce-818a-c77f5548e871 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:52.037406 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:52.099829 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8becc22b-fa4f-47ce-818a-c77f5548e871 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:52.137559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6b868-ca08-4d72-b3ef-62e78573cd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:52.138553 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6b868-ca08-4d72-b3ef-62e78573cd5b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:52.138842 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6b868-ca08-4d72-b3ef-62e78573cd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:52.139131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6b868-ca08-4d72-b3ef-62e78573cd5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:52.152152 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6b868-ca08-4d72-b3ef-62e78573cd5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:52.153715 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6b868-ca08-4d72-b3ef-62e78573cd5b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:52.171700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-1ea6b868-ca08-4d72-b3ef-62e78573cd5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:18:52.171700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:18:52.171700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:18:52.171700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:18:52.171700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:18:52.171700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:18:52.171700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:18:52.171700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:18:52.171700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:18:52.171700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:18:52.171700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:18:52.171700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:18:52.171700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:18:52.171700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:18:52.171700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:18:52.172639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:18:52.172639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:18:52.172639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:18:52.172639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:18:52.172639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:18:52.172639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:18:52.172639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:18:52.172639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:18:52.172639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:18:52.172639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:18:52.172639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:18:52.172639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:18:52.172639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:18:52.172639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:18:52.172639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:18:52.172639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:18:52.172639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:18:52.173589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:18:52.173589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:18:52.173589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:18:52.173589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:18:52.173813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 621/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:18:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:18:52.179969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05862fa-be44-4e80-a240-8e069960606b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:52.293669 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05862fa-be44-4e80-a240-8e069960606b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:52.334202 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac946282-d9cc-4c8f-8003-11cdcdb1b5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:52.334862 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac946282-d9cc-4c8f-8003-11cdcdb1b5ba None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:52.335145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac946282-d9cc-4c8f-8003-11cdcdb1b5ba None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:52.335420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac946282-d9cc-4c8f-8003-11cdcdb1b5ba None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:52.336137 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac946282-d9cc-4c8f-8003-11cdcdb1b5ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:18:52.336442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac946282-d9cc-4c8f-8003-11cdcdb1b5ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:52.337827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac946282-d9cc-4c8f-8003-11cdcdb1b5ba None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:52.349896 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 604/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:18:52.355608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6b868-ca08-4d72-b3ef-62e78573cd5b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:52.457304 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:52.464528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6b868-ca08-4d72-b3ef-62e78573cd5b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:52.500657 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756cb30-fbcd-45d9-9161-54905a38903e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:52.501560 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756cb30-fbcd-45d9-9161-54905a38903e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:52.501888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756cb30-fbcd-45d9-9161-54905a38903e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:52.502171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756cb30-fbcd-45d9-9161-54905a38903e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:52.510539 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756cb30-fbcd-45d9-9161-54905a38903e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:52.511977 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756cb30-fbcd-45d9-9161-54905a38903e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:52.526402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f756cb30-fbcd-45d9-9161-54905a38903e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:18:52.526402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:18:52.526402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:18:52.526402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:18:52.526402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:18:52.526402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:18:52.526402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:18:52.526402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:18:52.526402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:18:52.526402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:18:52.526402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:18:52.526402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:18:52.526402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:18:52.526402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:18:52.526402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:18:52.526402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:18:52.527549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:18:52.527549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:18:52.527549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:18:52.527549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:18:52.527549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:18:52.527549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:18:52.527549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:18:52.527549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:18:52.527549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:18:52.527549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:18:52.527549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:18:52.527549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:18:52.527549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:18:52.527549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:18:52.527549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:18:52.527549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:18:52.527549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:18:52.527549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:18:52.528585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:18:52.528585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:18:52.528585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:18:52.528585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:18:52.528585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:18:52.528585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:18:52.528585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:18:52.528585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:18:52.528585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:18:52.528585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:18:52.528585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 622/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:18:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:18:52.533792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac946282-d9cc-4c8f-8003-11cdcdb1b5ba None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:52.583247 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:52.645343 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac946282-d9cc-4c8f-8003-11cdcdb1b5ba None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:52.680784 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050b44f-d099-4065-82ec-4b7ee8e6d281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:52.681474 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050b44f-d099-4065-82ec-4b7ee8e6d281 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:52.681796 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050b44f-d099-4065-82ec-4b7ee8e6d281 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:52.682170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050b44f-d099-4065-82ec-4b7ee8e6d281 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:52.683305 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050b44f-d099-4065-82ec-4b7ee8e6d281 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:18:52.683641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050b44f-d099-4065-82ec-4b7ee8e6d281 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:52.685734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050b44f-d099-4065-82ec-4b7ee8e6d281 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:52.707497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 605/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Jul 18 10:18:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:18:52.713409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756cb30-fbcd-45d9-9161-54905a38903e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:52.821062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756cb30-fbcd-45d9-9161-54905a38903e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:52.858190 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2720af4-74f7-4590-9400-f3981a0acf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:52.859298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2720af4-74f7-4590-9400-f3981a0acf00 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:52.859773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2720af4-74f7-4590-9400-f3981a0acf00 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:52.860062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2720af4-74f7-4590-9400-f3981a0acf00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:52.871901 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2720af4-74f7-4590-9400-f3981a0acf00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:52.873405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2720af4-74f7-4590-9400-f3981a0acf00 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:52.892345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d2720af4-74f7-4590-9400-f3981a0acf00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:18:52.892345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:18:52.892345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:18:52.892345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:18:52.892345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:18:52.892345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:18:52.892345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:18:52.892345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:18:52.892345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:18:52.892345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:18:52.892345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:18:52.892345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:18:52.892345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:18:52.892345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:18:52.892345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:18:52.893500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:18:52.893500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:18:52.893500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:18:52.893500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:18:52.893500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:18:52.893500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:18:52.893500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:18:52.893500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:18:52.893500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:18:52.893500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:18:52.893500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:18:52.893500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:18:52.893500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:18:52.893500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:18:52.893500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:18:52.893500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:18:52.893500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:18:52.894421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:18:52.894421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:18:52.894421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:18:52.894421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:18:52.894421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 623/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:18:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:18:52.899759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050b44f-d099-4065-82ec-4b7ee8e6d281 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:53.015105 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050b44f-d099-4065-82ec-4b7ee8e6d281 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:53.017835 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:53.053906 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff3002-1fdb-4c1b-b2ff-8a18d42250f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:53.054704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff3002-1fdb-4c1b-b2ff-8a18d42250f8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:53.054992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff3002-1fdb-4c1b-b2ff-8a18d42250f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:53.055277 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff3002-1fdb-4c1b-b2ff-8a18d42250f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:53.056087 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff3002-1fdb-4c1b-b2ff-8a18d42250f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:18:53.056395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff3002-1fdb-4c1b-b2ff-8a18d42250f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:53.057942 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff3002-1fdb-4c1b-b2ff-8a18d42250f8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:53.069954 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 606/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:18:53.075835 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2720af4-74f7-4590-9400-f3981a0acf00 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:53.086072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:53.192179 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2720af4-74f7-4590-9400-f3981a0acf00 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:53.229379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384b33d-9e15-4d15-835e-c78a87566f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:53.230207 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384b33d-9e15-4d15-835e-c78a87566f10 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:53.230492 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384b33d-9e15-4d15-835e-c78a87566f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:53.230771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384b33d-9e15-4d15-835e-c78a87566f10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:53.240007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384b33d-9e15-4d15-835e-c78a87566f10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:53.241675 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384b33d-9e15-4d15-835e-c78a87566f10 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:53.257830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b384b33d-9e15-4d15-835e-c78a87566f10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:18:53.257830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:18:53.257830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:18:53.257830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:18:53.257830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:18:53.257830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:18:53.257830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:18:53.257830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:18:53.257830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:18:53.257830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:18:53.257830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:18:53.257830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:18:53.257830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:18:53.257830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:18:53.257830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:18:53.259029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:18:53.259029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:18:53.259029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:18:53.259029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:18:53.259029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:18:53.259029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:18:53.259029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:18:53.259029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:18:53.259029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:18:53.259029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:18:53.259029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:18:53.259029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:18:53.259029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:18:53.259029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:18:53.259029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:18:53.259029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:18:53.259029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:18:53.260225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:18:53.260225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:18:53.260225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:18:53.260225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:18:53.260225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:18:53.260225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:18:53.260225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:18:53.260225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:18:53.263932 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 624/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:18:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:18:53.265167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff3002-1fdb-4c1b-b2ff-8a18d42250f8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:53.377992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff3002-1fdb-4c1b-b2ff-8a18d42250f8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:53.415088 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039e0e9-237c-4137-b01b-f4560987891d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:53.415677 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039e0e9-237c-4137-b01b-f4560987891d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:53.415957 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039e0e9-237c-4137-b01b-f4560987891d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:53.416232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039e0e9-237c-4137-b01b-f4560987891d None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:53.417055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039e0e9-237c-4137-b01b-f4560987891d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:18:53.417363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039e0e9-237c-4137-b01b-f4560987891d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:53.418785 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039e0e9-237c-4137-b01b-f4560987891d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:53.432438 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 607/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:18:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:18:53.439162 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384b33d-9e15-4d15-835e-c78a87566f10 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:53.513199 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:53.550182 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384b33d-9e15-4d15-835e-c78a87566f10 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:53.588671 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d79d3-6a8f-4f10-8cba-a56ebfca875a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:53.589576 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d79d3-6a8f-4f10-8cba-a56ebfca875a None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:53.589862 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d79d3-6a8f-4f10-8cba-a56ebfca875a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:53.590142 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d79d3-6a8f-4f10-8cba-a56ebfca875a None admin] PATH_INFO: `/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/d07165f554a445169b2c3f0d173c0bb7/roles/f57b3e871ce142a090d6aa7dea9ac7d5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:53.607374 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d79d3-6a8f-4f10-8cba-a56ebfca875a None admin] RBAC: Authorizing `identity:create_grant(project_id=e71a3acd0adf46a2a35b965a27c4998f, user_id=d07165f554a445169b2c3f0d173c0bb7, role_id=f57b3e871ce142a090d6aa7dea9ac7d5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:53.609127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d79d3-6a8f-4f10-8cba-a56ebfca875a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:53.622978 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 625/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:18:53 2022] PUT /identity/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/d07165f554a445169b2c3f0d173c0bb7/roles/f57b3e871ce142a090d6aa7dea9ac7d5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:18:55.700433 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d5b14-8a8c-4bdd-9d4a-38770ccae748 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:55.701000 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d5b14-8a8c-4bdd-9d4a-38770ccae748 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:55.701000 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d5b14-8a8c-4bdd-9d4a-38770ccae748 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:55.702544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 608/1233] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:18:55.708126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f0239-622c-470f-84f2-26328b1e8171 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:55.708748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f0239-622c-470f-84f2-26328b1e8171 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:55.708748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f0239-622c-470f-84f2-26328b1e8171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:55.730192 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b72f0239-622c-470f-84f2-26328b1e8171 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:18:55.732210 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b72f0239-622c-470f-84f2-26328b1e8171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:18:55.812551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 626/1234] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:18:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:18:55.837574 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d5b14-8a8c-4bdd-9d4a-38770ccae748 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:55.865278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:55.946497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d5b14-8a8c-4bdd-9d4a-38770ccae748 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:55.983003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d4ac0-5823-4f6c-915a-468a54b335dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:55.983728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d4ac0-5823-4f6c-915a-468a54b335dc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:55.984039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d4ac0-5823-4f6c-915a-468a54b335dc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:55.984316 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d4ac0-5823-4f6c-915a-468a54b335dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:55.995959 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d4ac0-5823-4f6c-915a-468a54b335dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:18:56.003666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a00d4ac0-5823-4f6c-915a-468a54b335dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:18:56.003666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:18:56.003666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:18:56.003666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:18:56.003666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:18:56.003666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:18:56.003666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:18:56.003666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:18:56.003666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:18:56.003666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:18:56.003666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:18:56.003666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:18:56.003666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:18:56.003666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:18:56.003666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:18:56.003666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:18:56.005218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:18:56.005218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:18:56.005218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:18:56.005218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:18:56.005218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:18:56.005218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:18:56.005218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:18:56.005218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:18:56.005218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:18:56.005218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:18:56.005218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:18:56.005218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:18:56.005218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:18:56.005218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:18:56.005218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:18:56.005218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:18:56.006095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:18:56.006095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:18:56.006095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:18:56.006095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:18:56.006095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:18:56.006095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:18:56.006095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:18:56.006095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:18:56.006095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:18:56.006095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:18:56.006095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 10:18:56.006095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:18:56.006095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 609/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jul 18 10:18:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:18:56.011751 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f0239-622c-470f-84f2-26328b1e8171 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:56.061731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:56.122922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f0239-622c-470f-84f2-26328b1e8171 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:56.157237 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219ef71-ceae-4779-8799-6ee3be9230e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:56.157911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219ef71-ceae-4779-8799-6ee3be9230e2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:56.158194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219ef71-ceae-4779-8799-6ee3be9230e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:56.158470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219ef71-ceae-4779-8799-6ee3be9230e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:56.159223 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2219ef71-ceae-4779-8799-6ee3be9230e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:18:56.159531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2219ef71-ceae-4779-8799-6ee3be9230e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:56.160968 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2219ef71-ceae-4779-8799-6ee3be9230e2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:56.169798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 627/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jul 18 10:18:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:18:56.175770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d4ac0-5823-4f6c-915a-468a54b335dc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:56.266146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:56.286747 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d4ac0-5823-4f6c-915a-468a54b335dc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:56.323331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011cc7b3-af3b-4e03-8e07-d5273bbe5dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:56.324117 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cc7b3-af3b-4e03-8e07-d5273bbe5dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:56.324403 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cc7b3-af3b-4e03-8e07-d5273bbe5dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:56.324716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cc7b3-af3b-4e03-8e07-d5273bbe5dd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:56.336106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011cc7b3-af3b-4e03-8e07-d5273bbe5dd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:56.337618 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011cc7b3-af3b-4e03-8e07-d5273bbe5dd7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:56.356992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-011cc7b3-af3b-4e03-8e07-d5273bbe5dd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:18:56.356992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:18:56.356992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:18:56.356992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:18:56.356992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:18:56.356992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:18:56.356992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:18:56.356992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:18:56.356992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:18:56.356992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:18:56.356992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:18:56.356992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:18:56.356992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:18:56.356992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:18:56.356992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:18:56.358145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:18:56.358145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:18:56.358145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:18:56.358145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:18:56.358145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:18:56.358145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:18:56.358145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:18:56.358145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:18:56.358145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:18:56.358145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:18:56.358145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:18:56.358145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:18:56.358145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:18:56.358145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:18:56.358145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:18:56.358145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:18:56.358145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:18:56.359265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:18:56.359265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:18:56.359265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:18:56.359265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:18:56.365625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 610/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:18:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:18:56.365749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219ef71-ceae-4779-8799-6ee3be9230e2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:56.475284 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219ef71-ceae-4779-8799-6ee3be9230e2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:56.485678 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:56.515199 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818274b-fe33-413f-a7c7-b2bd11047bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:56.516031 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818274b-fe33-413f-a7c7-b2bd11047bcd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:56.517157 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818274b-fe33-413f-a7c7-b2bd11047bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:56.517460 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818274b-fe33-413f-a7c7-b2bd11047bcd None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:56.518328 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818274b-fe33-413f-a7c7-b2bd11047bcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:18:56.518647 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818274b-fe33-413f-a7c7-b2bd11047bcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:56.520129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818274b-fe33-413f-a7c7-b2bd11047bcd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:56.534140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 628/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:18:56.540317 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011cc7b3-af3b-4e03-8e07-d5273bbe5dd7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:56.653941 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011cc7b3-af3b-4e03-8e07-d5273bbe5dd7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:56.690051 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2174427-23c8-4d1a-a5bb-edaef515519a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:56.690843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2174427-23c8-4d1a-a5bb-edaef515519a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:56.691179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2174427-23c8-4d1a-a5bb-edaef515519a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:56.691483 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2174427-23c8-4d1a-a5bb-edaef515519a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:56.694398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:56.701589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2174427-23c8-4d1a-a5bb-edaef515519a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:56.703045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2174427-23c8-4d1a-a5bb-edaef515519a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:56.717346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b2174427-23c8-4d1a-a5bb-edaef515519a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:18:56.717346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:18:56.717346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:18:56.717346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:18:56.717346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:18:56.717346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:18:56.717346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:18:56.717346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:18:56.717346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:18:56.717346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:18:56.717346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:18:56.717346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:18:56.717346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:18:56.717346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:18:56.717346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:18:56.717346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:18:56.718507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:18:56.718507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:18:56.718507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:18:56.718507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:18:56.718507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:18:56.718507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:18:56.718507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:18:56.718507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:18:56.718507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:18:56.718507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:18:56.718507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:18:56.718507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:18:56.718507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:18:56.718507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:18:56.718507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:18:56.718507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:18:56.718507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:18:56.718507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:18:56.719533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:18:56.719533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:18:56.719533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:18:56.719533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:18:56.719533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:18:56.719533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:18:56.719533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:18:56.719533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:18:56.719533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:18:56.719533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:18:56.719533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 611/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:18:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:18:56.725349 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818274b-fe33-413f-a7c7-b2bd11047bcd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:56.842410 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818274b-fe33-413f-a7c7-b2bd11047bcd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:56.883202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289ee9d-f966-4e2c-8ffe-ae4ea20a739d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:56.883868 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289ee9d-f966-4e2c-8ffe-ae4ea20a739d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:56.884153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289ee9d-f966-4e2c-8ffe-ae4ea20a739d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:56.884475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289ee9d-f966-4e2c-8ffe-ae4ea20a739d None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:56.885345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289ee9d-f966-4e2c-8ffe-ae4ea20a739d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:18:56.885651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289ee9d-f966-4e2c-8ffe-ae4ea20a739d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:56.887067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289ee9d-f966-4e2c-8ffe-ae4ea20a739d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:56.906308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 629/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Jul 18 10:18:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:18:56.913352 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2174427-23c8-4d1a-a5bb-edaef515519a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:57.026466 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2174427-23c8-4d1a-a5bb-edaef515519a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:57.065546 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b9e6f-14b8-42bf-9651-049f9d1cc219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:57.066193 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b9e6f-14b8-42bf-9651-049f9d1cc219 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:57.066521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b9e6f-14b8-42bf-9651-049f9d1cc219 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:57.066802 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b9e6f-14b8-42bf-9651-049f9d1cc219 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:57.138950 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:57.146783 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b9e6f-14b8-42bf-9651-049f9d1cc219 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:57.148236 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b9e6f-14b8-42bf-9651-049f9d1cc219 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:57.261556 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 612/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:18:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:18:57.263091 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289ee9d-f966-4e2c-8ffe-ae4ea20a739d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:57.297847 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:57.384073 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289ee9d-f966-4e2c-8ffe-ae4ea20a739d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:57.423759 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360cb77-b4a3-4ce8-8fbb-c109a2fce761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:57.424477 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360cb77-b4a3-4ce8-8fbb-c109a2fce761 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:57.424833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360cb77-b4a3-4ce8-8fbb-c109a2fce761 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:57.425121 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360cb77-b4a3-4ce8-8fbb-c109a2fce761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:57.438560 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360cb77-b4a3-4ce8-8fbb-c109a2fce761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:57.440251 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360cb77-b4a3-4ce8-8fbb-c109a2fce761 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:57.463046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e360cb77-b4a3-4ce8-8fbb-c109a2fce761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:18:57.463046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:18:57.463046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:18:57.463046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:18:57.463046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:18:57.463046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:18:57.463046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:18:57.463046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:18:57.463046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:18:57.463046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:18:57.463046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:18:57.463046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:18:57.463046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:18:57.463046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:18:57.463046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:18:57.464296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:18:57.464296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:18:57.464296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:18:57.464296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:18:57.464296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:18:57.464296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:18:57.464296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:18:57.464296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:18:57.464296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:18:57.464296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:18:57.464296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:18:57.464296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:18:57.464296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:18:57.464296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:18:57.464296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:18:57.464296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:18:57.464296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:18:57.465627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:18:57.465627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:18:57.465627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:18:57.465627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:18:57.465627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 630/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:18:57.470633 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b9e6f-14b8-42bf-9651-049f9d1cc219 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:57.576464 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:57.583799 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b9e6f-14b8-42bf-9651-049f9d1cc219 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:57.623848 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd03262-1d29-42f0-9e83-2ffce46ec92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:57.624530 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd03262-1d29-42f0-9e83-2ffce46ec92c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:57.624812 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd03262-1d29-42f0-9e83-2ffce46ec92c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:57.625088 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd03262-1d29-42f0-9e83-2ffce46ec92c None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:57.625802 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd03262-1d29-42f0-9e83-2ffce46ec92c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:18:57.626115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd03262-1d29-42f0-9e83-2ffce46ec92c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:57.627466 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd03262-1d29-42f0-9e83-2ffce46ec92c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:57.642478 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 613/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:18:57.649931 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360cb77-b4a3-4ce8-8fbb-c109a2fce761 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:57.731257 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:18:57.758651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360cb77-b4a3-4ce8-8fbb-c109a2fce761 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:57.798391 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8cf89-9283-4371-b211-b31fb1a9daeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:57.799235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8cf89-9283-4371-b211-b31fb1a9daeb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:57.799777 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8cf89-9283-4371-b211-b31fb1a9daeb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:57.800084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8cf89-9283-4371-b211-b31fb1a9daeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:57.809438 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8cf89-9283-4371-b211-b31fb1a9daeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:57.811132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8cf89-9283-4371-b211-b31fb1a9daeb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:57.830537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-95e8cf89-9283-4371-b211-b31fb1a9daeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:18:57.830537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:18:57.830537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:18:57.830537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:18:57.830537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:18:57.830537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:18:57.830537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:18:57.830537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:18:57.830537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:18:57.830537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:18:57.830537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:18:57.830537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:18:57.830537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:18:57.830537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:18:57.830537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:18:57.831730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:18:57.831730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:18:57.831730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:18:57.831730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:18:57.831730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:18:57.831730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:18:57.831730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:18:57.831730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:18:57.831730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:18:57.831730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:18:57.831730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:18:57.831730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:18:57.831730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:18:57.831730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:18:57.831730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:18:57.831730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:18:57.831730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:18:57.832742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:18:57.832742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:18:57.832742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:18:57.832742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:18:57.832742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:18:57.832742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:18:57.832742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:18:57.832742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:18:57.832742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 631/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:18:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 10:18:57.838073 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd03262-1d29-42f0-9e83-2ffce46ec92c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:57.959760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd03262-1d29-42f0-9e83-2ffce46ec92c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:58.001929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b59ba1-e2d4-4e71-af5d-d0e126ce0bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:58.002507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b59ba1-e2d4-4e71-af5d-d0e126ce0bff None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:58.002773 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b59ba1-e2d4-4e71-af5d-d0e126ce0bff None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:58.003023 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b59ba1-e2d4-4e71-af5d-d0e126ce0bff None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:58.003821 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b59ba1-e2d4-4e71-af5d-d0e126ce0bff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:18:58.004715 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b59ba1-e2d4-4e71-af5d-d0e126ce0bff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:58.006232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b59ba1-e2d4-4e71-af5d-d0e126ce0bff None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:58.019448 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 614/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:18:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:18:58.025697 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8cf89-9283-4371-b211-b31fb1a9daeb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:18:58.135878 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8cf89-9283-4371-b211-b31fb1a9daeb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:18:58.175711 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4750a2a-1bad-4544-9c5b-b0a556f35164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:18:58.177072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4750a2a-1bad-4544-9c5b-b0a556f35164 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:18:58.177358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4750a2a-1bad-4544-9c5b-b0a556f35164 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:18:58.177635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4750a2a-1bad-4544-9c5b-b0a556f35164 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:18:58.178477 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4750a2a-1bad-4544-9c5b-b0a556f35164 None admin] RBAC: Adding query filter params (role.id=f57b3e871ce142a090d6aa7dea9ac7d5, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=d07165f554a445169b2c3f0d173c0bb7) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:18:58.178793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4750a2a-1bad-4544-9c5b-b0a556f35164 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:18:58.180241 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4750a2a-1bad-4544-9c5b-b0a556f35164 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:18:58.188271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 632/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:18:58 2022] GET /identity/v3/role_assignments?user.id=d07165f554a445169b2c3f0d173c0bb7&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=f57b3e871ce142a090d6aa7dea9ac7d5 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:19:00.280716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ead59-3342-470d-8d98-536b8714b067 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:00.281241 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ead59-3342-470d-8d98-536b8714b067 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:00.281351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ead59-3342-470d-8d98-536b8714b067 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:00.282644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 615/1247] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:19:00.288773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f0a84-7a57-42c3-8379-ff084f957e5c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:00.289193 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f0a84-7a57-42c3-8379-ff084f957e5c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:00.289346 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f0a84-7a57-42c3-8379-ff084f957e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:00.295258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:00.318638 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-400f0a84-7a57-42c3-8379-ff084f957e5c None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:19:00.321830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-400f0a84-7a57-42c3-8379-ff084f957e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:19:00.402078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 633/1248] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:19:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:19:00.425035 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ead59-3342-470d-8d98-536b8714b067 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:00.444177 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:00.540988 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ead59-3342-470d-8d98-536b8714b067 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:00.576700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec818b-d1a9-405c-8580-e292dc31f64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:00.577494 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec818b-d1a9-405c-8580-e292dc31f64d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:00.577780 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec818b-d1a9-405c-8580-e292dc31f64d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:00.578054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec818b-d1a9-405c-8580-e292dc31f64d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:00.589741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec818b-d1a9-405c-8580-e292dc31f64d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:19:00.599179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-2fec818b-d1a9-405c-8580-e292dc31f64d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:19:00.599179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:00.599179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:19:00.599179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:19:00.599179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:19:00.599179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:19:00.599179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:19:00.599179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:19:00.599179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:19:00.599179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:19:00.599179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:19:00.599179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:19:00.599179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:19:00.599179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:19:00.599179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:19:00.599179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:19:00.600596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:19:00.600596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:19:00.600596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:19:00.600596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:19:00.600596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:19:00.600596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:19:00.600596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:19:00.600596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:19:00.600596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:19:00.600596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:19:00.600596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:19:00.600596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:19:00.600596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:19:00.600596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:19:00.600596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:19:00.600596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:19:00.601679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:19:00.601679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:19:00.601679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:19:00.601679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:19:00.601679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:19:00.601679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:19:00.601679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:19:00.601679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:19:00.601679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:19:00.601679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:19:00.601679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:19:00.601679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:19:00.602753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 616/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:19:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:19:00.607923 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f0a84-7a57-42c3-8379-ff084f957e5c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:00.724435 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f0a84-7a57-42c3-8379-ff084f957e5c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:00.761119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:00.767493 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eabb0f-530f-41d5-860e-9b0a19401715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:00.768400 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eabb0f-530f-41d5-860e-9b0a19401715 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:00.768798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eabb0f-530f-41d5-860e-9b0a19401715 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:00.769123 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eabb0f-530f-41d5-860e-9b0a19401715 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:00.770093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eabb0f-530f-41d5-860e-9b0a19401715 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:00.770750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eabb0f-530f-41d5-860e-9b0a19401715 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:00.772529 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eabb0f-530f-41d5-860e-9b0a19401715 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:00.782335 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 634/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:19:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:00.788180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec818b-d1a9-405c-8580-e292dc31f64d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:00.878490 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:00.904168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec818b-d1a9-405c-8580-e292dc31f64d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:00.943831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95c196-0a8c-4f95-85f7-4ed8c88fee04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:00.944899 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95c196-0a8c-4f95-85f7-4ed8c88fee04 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:00.945180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95c196-0a8c-4f95-85f7-4ed8c88fee04 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:00.945616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95c196-0a8c-4f95-85f7-4ed8c88fee04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:00.958237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95c196-0a8c-4f95-85f7-4ed8c88fee04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:00.959930 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95c196-0a8c-4f95-85f7-4ed8c88fee04 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:00.978174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7b95c196-0a8c-4f95-85f7-4ed8c88fee04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:19:00.978174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:00.978174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:19:00.978174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:19:00.978174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:19:00.978174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:19:00.978174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:19:00.978174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:19:00.978174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:19:00.978174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:19:00.978174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:19:00.978174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:19:00.978174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:19:00.978174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:19:00.978174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:19:00.979509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:19:00.979509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:19:00.979509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:19:00.979509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:19:00.979509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:19:00.979509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:19:00.979509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:19:00.979509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:19:00.979509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:19:00.979509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:19:00.979509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:19:00.979509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:19:00.979509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:19:00.979509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:19:00.979509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:19:00.979509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:19:00.979509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:19:00.980843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:19:00.980843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:19:00.980843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:19:00.980843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:19:00.980843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 617/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:19:00.985500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eabb0f-530f-41d5-860e-9b0a19401715 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:01.103582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eabb0f-530f-41d5-860e-9b0a19401715 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:01.143173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3c177-fb32-42ed-a264-459432bddb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:01.143847 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3c177-fb32-42ed-a264-459432bddb65 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:01.144135 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3c177-fb32-42ed-a264-459432bddb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:01.144418 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3c177-fb32-42ed-a264-459432bddb65 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:01.145265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff3c177-fb32-42ed-a264-459432bddb65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:01.145577 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff3c177-fb32-42ed-a264-459432bddb65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:01.146952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff3c177-fb32-42ed-a264-459432bddb65 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:01.159572 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 635/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:01.165893 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95c196-0a8c-4f95-85f7-4ed8c88fee04 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:01.278875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95c196-0a8c-4f95-85f7-4ed8c88fee04 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:01.309574 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:01.319701 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b5a6a-def6-4a40-8905-e94756a538ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:01.320532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b5a6a-def6-4a40-8905-e94756a538ae None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:01.321032 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b5a6a-def6-4a40-8905-e94756a538ae None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:01.321421 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b5a6a-def6-4a40-8905-e94756a538ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:01.334066 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b5a6a-def6-4a40-8905-e94756a538ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:01.336067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b5a6a-def6-4a40-8905-e94756a538ae None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:01.353203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d44b5a6a-def6-4a40-8905-e94756a538ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:19:01.353203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:01.353203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:19:01.353203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:19:01.353203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:19:01.353203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:19:01.353203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:19:01.353203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:19:01.353203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:19:01.353203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:19:01.353203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:19:01.353203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:19:01.353203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:19:01.353203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:19:01.353203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:19:01.353203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:19:01.354553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:19:01.354553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:19:01.354553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:19:01.354553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:19:01.354553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:19:01.354553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:19:01.354553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:19:01.354553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:19:01.354553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:19:01.354553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:19:01.354553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:19:01.354553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:19:01.354553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:19:01.354553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:19:01.354553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:19:01.354553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:19:01.354553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:19:01.354553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:19:01.356108 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:19:01.356108 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:19:01.356108 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:19:01.356108 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:19:01.356108 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:19:01.356108 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:19:01.356108 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:19:01.356108 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:19:01.356108 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:19:01.356108 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:19:01.356108 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 618/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:19:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:19:01.361149 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3c177-fb32-42ed-a264-459432bddb65 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:01.423738 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:01.476703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3c177-fb32-42ed-a264-459432bddb65 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:01.515856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d08870-4d2a-4c3c-8d83-601a85eba2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:01.516620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d08870-4d2a-4c3c-8d83-601a85eba2fe None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:01.516902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d08870-4d2a-4c3c-8d83-601a85eba2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:01.517187 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d08870-4d2a-4c3c-8d83-601a85eba2fe None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:01.518177 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d08870-4d2a-4c3c-8d83-601a85eba2fe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:01.518527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d08870-4d2a-4c3c-8d83-601a85eba2fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:01.520121 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d08870-4d2a-4c3c-8d83-601a85eba2fe None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:01.539341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 636/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Jul 18 10:19:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:01.545211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b5a6a-def6-4a40-8905-e94756a538ae None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:01.655997 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b5a6a-def6-4a40-8905-e94756a538ae None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:01.694515 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8b1a5-061d-43a1-b508-9feeb1fd9080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:01.695222 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8b1a5-061d-43a1-b508-9feeb1fd9080 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:01.695574 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8b1a5-061d-43a1-b508-9feeb1fd9080 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:01.695876 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8b1a5-061d-43a1-b508-9feeb1fd9080 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:01.758653 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:01.772561 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8b1a5-061d-43a1-b508-9feeb1fd9080 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:01.774162 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8b1a5-061d-43a1-b508-9feeb1fd9080 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:01.899531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 619/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:19:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:19:01.905949 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d08870-4d2a-4c3c-8d83-601a85eba2fe None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:02.019726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d08870-4d2a-4c3c-8d83-601a85eba2fe None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:02.036050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:02.060662 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ae99e-1b6e-4b19-b38c-a8f5dcb601cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:02.061531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ae99e-1b6e-4b19-b38c-a8f5dcb601cc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:02.061836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ae99e-1b6e-4b19-b38c-a8f5dcb601cc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:02.062189 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ae99e-1b6e-4b19-b38c-a8f5dcb601cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:02.073090 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ae99e-1b6e-4b19-b38c-a8f5dcb601cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:02.074535 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ae99e-1b6e-4b19-b38c-a8f5dcb601cc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:02.092063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d13ae99e-1b6e-4b19-b38c-a8f5dcb601cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:19:02.092063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:02.092063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:19:02.092063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:19:02.092063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:19:02.092063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:19:02.092063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:19:02.092063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:19:02.092063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:19:02.092063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:19:02.092063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:19:02.092063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:19:02.092063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:19:02.092063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:19:02.092063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:19:02.093262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:19:02.093262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:19:02.093262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:19:02.093262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:19:02.093262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:19:02.093262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:19:02.093262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:19:02.093262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:19:02.093262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:19:02.093262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:19:02.093262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:19:02.093262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:19:02.093262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:19:02.093262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:19:02.093262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:19:02.093262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:19:02.093262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:19:02.094303 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:19:02.094303 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:19:02.094303 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:19:02.094303 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:19:02.094303 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 637/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:19:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:19:02.099541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8b1a5-061d-43a1-b508-9feeb1fd9080 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:02.205702 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:02.218995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8b1a5-061d-43a1-b508-9feeb1fd9080 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:02.260026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df4e65-7946-4487-b206-a40e0accd003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:02.262891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df4e65-7946-4487-b206-a40e0accd003 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:02.262891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df4e65-7946-4487-b206-a40e0accd003 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:02.262891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df4e65-7946-4487-b206-a40e0accd003 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:02.262891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df4e65-7946-4487-b206-a40e0accd003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:02.262891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df4e65-7946-4487-b206-a40e0accd003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:02.263661 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df4e65-7946-4487-b206-a40e0accd003 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:02.276455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 620/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:02.282270 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ae99e-1b6e-4b19-b38c-a8f5dcb601cc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:02.392142 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ae99e-1b6e-4b19-b38c-a8f5dcb601cc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:02.430448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76231c-cc6c-4d97-9c33-274b987462f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:02.431118 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76231c-cc6c-4d97-9c33-274b987462f1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:02.431411 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76231c-cc6c-4d97-9c33-274b987462f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:02.431698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76231c-cc6c-4d97-9c33-274b987462f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:02.438600 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76231c-cc6c-4d97-9c33-274b987462f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:02.440191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76231c-cc6c-4d97-9c33-274b987462f1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:02.455140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-3b76231c-cc6c-4d97-9c33-274b987462f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:19:02.455140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:02.455140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:19:02.455140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:19:02.455140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:19:02.455140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:19:02.455140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:19:02.455140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:19:02.455140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:19:02.455140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:19:02.455140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:19:02.455140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:19:02.455140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:19:02.455140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:19:02.455140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:19:02.456299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:19:02.456299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:19:02.456299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:19:02.456299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:19:02.456299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:19:02.456299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:19:02.456299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:19:02.456299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:19:02.456299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:19:02.456299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:19:02.456299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:19:02.456299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:19:02.456299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:19:02.456299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:19:02.456299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:19:02.456299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:19:02.456299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:19:02.458592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:19:02.458592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:19:02.458592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:19:02.458592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:19:02.458592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:19:02.458592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:19:02.458592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:19:02.458592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:19:02.458592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 638/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:19:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:19:02.463097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df4e65-7946-4487-b206-a40e0accd003 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:02.570071 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df4e65-7946-4487-b206-a40e0accd003 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:02.605435 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fbf3d-ebc8-421e-8ebb-fe31e0fb2e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:02.606229 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fbf3d-ebc8-421e-8ebb-fe31e0fb2e66 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:02.606796 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fbf3d-ebc8-421e-8ebb-fe31e0fb2e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:02.607105 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fbf3d-ebc8-421e-8ebb-fe31e0fb2e66 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:02.607911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fbf3d-ebc8-421e-8ebb-fe31e0fb2e66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:02.608220 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fbf3d-ebc8-421e-8ebb-fe31e0fb2e66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:02.609762 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fbf3d-ebc8-421e-8ebb-fe31e0fb2e66 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:02.623339 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 621/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:19:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:02.629911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76231c-cc6c-4d97-9c33-274b987462f1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:02.658010 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:02.738376 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76231c-cc6c-4d97-9c33-274b987462f1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:02.775416 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7dbf0-0362-42d1-9e2b-6d72de45546e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:02.776074 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7dbf0-0362-42d1-9e2b-6d72de45546e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:02.776358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7dbf0-0362-42d1-9e2b-6d72de45546e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:02.776665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7dbf0-0362-42d1-9e2b-6d72de45546e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:02.777503 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7dbf0-0362-42d1-9e2b-6d72de45546e None admin] RBAC: Adding query filter params (role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=d07165f554a445169b2c3f0d173c0bb7) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:02.777813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7dbf0-0362-42d1-9e2b-6d72de45546e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:02.779237 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7dbf0-0362-42d1-9e2b-6d72de45546e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:02.785352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 639/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:19:02 2022] GET /identity/v3/role_assignments?user.id=d07165f554a445169b2c3f0d173c0bb7&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:04.793508 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36039f-0df0-4711-a207-29bf6c902c28 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:04.794075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36039f-0df0-4711-a207-29bf6c902c28 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:04.794209 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36039f-0df0-4711-a207-29bf6c902c28 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:04.796548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 622/1261] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:19:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:19:04.802407 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca39a3-9d09-495f-958e-d7e040a56d21 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:04.802896 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca39a3-9d09-495f-958e-d7e040a56d21 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:04.803016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca39a3-9d09-495f-958e-d7e040a56d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:04.822613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ccca39a3-9d09-495f-958e-d7e040a56d21 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:19:04.823963 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ccca39a3-9d09-495f-958e-d7e040a56d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:19:04.890713 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 640/1262] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:19:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:19:04.913392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36039f-0df0-4711-a207-29bf6c902c28 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:04.923912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:05.014625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36039f-0df0-4711-a207-29bf6c902c28 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:05.048765 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5dca5-2c18-4c56-9e72-4a72311872ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:05.049734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5dca5-2c18-4c56-9e72-4a72311872ab None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:05.050017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5dca5-2c18-4c56-9e72-4a72311872ab None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:05.050293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5dca5-2c18-4c56-9e72-4a72311872ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:05.060752 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5dca5-2c18-4c56-9e72-4a72311872ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:19:05.068249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ddd5dca5-2c18-4c56-9e72-4a72311872ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:19:05.068249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:05.068249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:19:05.068249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:19:05.068249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:19:05.068249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:19:05.068249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:19:05.068249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:19:05.068249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:19:05.068249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:19:05.068249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:19:05.068249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:19:05.068249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:19:05.068249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:19:05.068249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:19:05.068249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:19:05.069459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:19:05.069459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:19:05.069459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:19:05.069459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:19:05.069459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:19:05.069459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:19:05.069459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:19:05.069459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:19:05.069459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:19:05.069459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:19:05.069459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:19:05.069459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:19:05.069459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:19:05.069459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:19:05.069459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:19:05.069459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:19:05.070529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:19:05.070529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:19:05.070529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:19:05.070529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:19:05.070529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:19:05.070529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:19:05.070529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:19:05.070529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:19:05.070529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:19:05.070529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:19:05.070529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:19:05.070529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:19:05.070529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 623/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:19:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:19:05.075461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca39a3-9d09-495f-958e-d7e040a56d21 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:05.093189 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:05.172790 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca39a3-9d09-495f-958e-d7e040a56d21 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:05.208435 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99691668-bbd8-4cf2-979c-6d1b771f1461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:05.209126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99691668-bbd8-4cf2-979c-6d1b771f1461 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:05.209408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99691668-bbd8-4cf2-979c-6d1b771f1461 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:05.209683 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99691668-bbd8-4cf2-979c-6d1b771f1461 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:05.210424 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99691668-bbd8-4cf2-979c-6d1b771f1461 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:05.210731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99691668-bbd8-4cf2-979c-6d1b771f1461 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:05.212115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99691668-bbd8-4cf2-979c-6d1b771f1461 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:05.219894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 641/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:19:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:05.225596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5dca5-2c18-4c56-9e72-4a72311872ab None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:05.290968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:05.323065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5dca5-2c18-4c56-9e72-4a72311872ab None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:05.355744 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a6166-4750-4ea3-b5b5-944e80f1b6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:05.356799 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a6166-4750-4ea3-b5b5-944e80f1b6e1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:05.357081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a6166-4750-4ea3-b5b5-944e80f1b6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:05.357354 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a6166-4750-4ea3-b5b5-944e80f1b6e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:05.368001 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a6166-4750-4ea3-b5b5-944e80f1b6e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:05.369467 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a6166-4750-4ea3-b5b5-944e80f1b6e1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:05.386330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-211a6166-4750-4ea3-b5b5-944e80f1b6e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:19:05.386330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:05.386330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:19:05.386330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:19:05.386330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:19:05.386330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:19:05.386330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:19:05.386330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:19:05.386330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:19:05.386330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:19:05.386330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:19:05.386330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:19:05.386330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:19:05.386330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:19:05.386330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:19:05.387388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:19:05.387388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:19:05.387388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:19:05.387388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:19:05.387388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:19:05.387388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:19:05.387388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:19:05.387388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:19:05.387388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:19:05.387388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:19:05.387388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:19:05.387388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:19:05.387388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:19:05.387388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:19:05.387388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:19:05.387388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:19:05.387388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:19:05.388397 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:19:05.388397 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:19:05.388397 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:19:05.388397 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:19:05.388397 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 624/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:19:05.393315 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99691668-bbd8-4cf2-979c-6d1b771f1461 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:05.471341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:05.490753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99691668-bbd8-4cf2-979c-6d1b771f1461 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:05.522863 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6a25b-9d5c-48f3-9959-12f1541b1be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:05.523510 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6a25b-9d5c-48f3-9959-12f1541b1be8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:05.523798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6a25b-9d5c-48f3-9959-12f1541b1be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:05.524076 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6a25b-9d5c-48f3-9959-12f1541b1be8 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:05.524826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6a25b-9d5c-48f3-9959-12f1541b1be8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:05.525134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6a25b-9d5c-48f3-9959-12f1541b1be8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:05.526496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6a25b-9d5c-48f3-9959-12f1541b1be8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:05.537758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 642/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:05.543268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a6166-4750-4ea3-b5b5-944e80f1b6e1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:05.639402 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a6166-4750-4ea3-b5b5-944e80f1b6e1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:05.657727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:05.671643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cf0ea-b89b-4bb7-9ef5-98d058fca2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:05.672427 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cf0ea-b89b-4bb7-9ef5-98d058fca2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:05.672736 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cf0ea-b89b-4bb7-9ef5-98d058fca2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:05.673008 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cf0ea-b89b-4bb7-9ef5-98d058fca2d1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:05.680643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cf0ea-b89b-4bb7-9ef5-98d058fca2d1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:05.682069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cf0ea-b89b-4bb7-9ef5-98d058fca2d1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:05.695786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a74cf0ea-b89b-4bb7-9ef5-98d058fca2d1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:19:05.695786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:05.695786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:19:05.695786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:19:05.695786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:19:05.695786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:19:05.695786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:19:05.695786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:19:05.695786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:19:05.695786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:19:05.695786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:19:05.695786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:19:05.695786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:19:05.695786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:19:05.695786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:19:05.695786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:19:05.696766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:19:05.696766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:19:05.696766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:19:05.696766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:19:05.696766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:19:05.696766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:19:05.696766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:19:05.696766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:19:05.696766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:19:05.696766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:19:05.696766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:19:05.696766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:19:05.696766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:19:05.696766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:19:05.696766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:19:05.696766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:19:05.696766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:19:05.696766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:19:05.698030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:19:05.698030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:19:05.698030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:19:05.698030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:19:05.698030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:19:05.698030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:19:05.698030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:19:05.698030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:19:05.698030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:19:05.698030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:19:05.698030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 625/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:19:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:19:05.702588 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6a25b-9d5c-48f3-9959-12f1541b1be8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:05.799502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6a25b-9d5c-48f3-9959-12f1541b1be8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:05.840265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c8a29-0d25-46e8-8b6b-1dacac56599d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:05.841208 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c8a29-0d25-46e8-8b6b-1dacac56599d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:05.841505 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c8a29-0d25-46e8-8b6b-1dacac56599d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:05.841898 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c8a29-0d25-46e8-8b6b-1dacac56599d None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:05.842958 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c8a29-0d25-46e8-8b6b-1dacac56599d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:05.843434 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c8a29-0d25-46e8-8b6b-1dacac56599d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:05.845260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c8a29-0d25-46e8-8b6b-1dacac56599d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:05.855773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:05.865090 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 643/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Jul 18 10:19:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:05.870957 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cf0ea-b89b-4bb7-9ef5-98d058fca2d1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:05.966527 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cf0ea-b89b-4bb7-9ef5-98d058fca2d1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:05.998702 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12865787-3450-4c69-98dc-d8c58d1ddc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:05.999471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12865787-3450-4c69-98dc-d8c58d1ddc6c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:05.999756 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12865787-3450-4c69-98dc-d8c58d1ddc6c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:06.000032 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12865787-3450-4c69-98dc-d8c58d1ddc6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:06.010664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12865787-3450-4c69-98dc-d8c58d1ddc6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:06.012094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12865787-3450-4c69-98dc-d8c58d1ddc6c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:06.028541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-12865787-3450-4c69-98dc-d8c58d1ddc6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:19:06.028541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:06.028541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:19:06.028541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:19:06.028541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:19:06.028541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:19:06.028541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:19:06.028541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:19:06.028541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:19:06.028541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:19:06.028541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:19:06.028541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:19:06.028541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:19:06.028541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:19:06.028541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:19:06.029625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:19:06.029625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:19:06.029625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:19:06.029625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:19:06.029625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:19:06.029625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:19:06.029625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:19:06.029625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:19:06.029625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:19:06.029625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:19:06.029625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:19:06.029625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:19:06.029625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:19:06.029625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:19:06.029625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:19:06.029625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:19:06.029625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:19:06.030944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:19:06.030944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:19:06.030944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:19:06.030944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:19:06.030944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 626/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:19:06.035648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c8a29-0d25-46e8-8b6b-1dacac56599d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:06.134043 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c8a29-0d25-46e8-8b6b-1dacac56599d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:06.166442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2f1c2-1c35-4ef0-9023-5c902aa197d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:06.167102 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2f1c2-1c35-4ef0-9023-5c902aa197d0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:06.167391 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2f1c2-1c35-4ef0-9023-5c902aa197d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:06.167672 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2f1c2-1c35-4ef0-9023-5c902aa197d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:06.168398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2f1c2-1c35-4ef0-9023-5c902aa197d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:06.168742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2f1c2-1c35-4ef0-9023-5c902aa197d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:06.170115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2f1c2-1c35-4ef0-9023-5c902aa197d0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:06.181645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 644/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:06.187319 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12865787-3450-4c69-98dc-d8c58d1ddc6c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:06.218258 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:06.283991 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12865787-3450-4c69-98dc-d8c58d1ddc6c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:06.315664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4264f0a-c821-45af-ac84-def5b264c604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:06.316462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4264f0a-c821-45af-ac84-def5b264c604 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:06.316772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4264f0a-c821-45af-ac84-def5b264c604 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:06.317044 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4264f0a-c821-45af-ac84-def5b264c604 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:06.323402 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4264f0a-c821-45af-ac84-def5b264c604 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:06.324860 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4264f0a-c821-45af-ac84-def5b264c604 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:06.339097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b4264f0a-c821-45af-ac84-def5b264c604 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:19:06.339097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:06.339097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:19:06.339097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:19:06.339097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:19:06.339097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:19:06.339097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:19:06.339097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:19:06.339097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:19:06.339097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:19:06.339097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:19:06.339097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:19:06.339097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:19:06.339097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:19:06.339097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:19:06.340172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:19:06.340172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:19:06.340172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:19:06.340172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:19:06.340172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:19:06.340172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:19:06.340172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:19:06.340172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:19:06.340172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:19:06.340172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:19:06.340172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:19:06.340172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:19:06.340172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:19:06.340172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:19:06.340172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:19:06.340172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:19:06.340172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:19:06.341514 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:19:06.341514 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:19:06.341514 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:19:06.341514 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:19:06.341514 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:19:06.341514 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:19:06.341514 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:19:06.341514 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:19:06.341514 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 627/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:19:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:19:06.346088 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2f1c2-1c35-4ef0-9023-5c902aa197d0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:06.405681 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:06.445648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2f1c2-1c35-4ef0-9023-5c902aa197d0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:06.477605 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff325f-82ba-4b85-8172-11a55c95847d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:06.478187 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff325f-82ba-4b85-8172-11a55c95847d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:06.478468 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff325f-82ba-4b85-8172-11a55c95847d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:06.478740 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff325f-82ba-4b85-8172-11a55c95847d None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:06.479520 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff325f-82ba-4b85-8172-11a55c95847d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:06.479828 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff325f-82ba-4b85-8172-11a55c95847d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:06.481276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff325f-82ba-4b85-8172-11a55c95847d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:06.493742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 645/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:19:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:06.499820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4264f0a-c821-45af-ac84-def5b264c604 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:06.588287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:06.597718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4264f0a-c821-45af-ac84-def5b264c604 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:06.630651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092aa69-13b7-47af-b368-aa3a6913fffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:06.631524 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092aa69-13b7-47af-b368-aa3a6913fffc None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:06.631805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092aa69-13b7-47af-b368-aa3a6913fffc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:06.632081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092aa69-13b7-47af-b368-aa3a6913fffc None admin] PATH_INFO: `/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/d07165f554a445169b2c3f0d173c0bb7/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:06.634045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092aa69-13b7-47af-b368-aa3a6913fffc None admin] RBAC: Authorizing `identity:create_grant(project_id=e71a3acd0adf46a2a35b965a27c4998f, user_id=d07165f554a445169b2c3f0d173c0bb7, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:06.635465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092aa69-13b7-47af-b368-aa3a6913fffc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:06.648119 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 628/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jul 18 10:19:06 2022] PUT /identity/v3/projects/e71a3acd0adf46a2a35b965a27c4998f/users/d07165f554a445169b2c3f0d173c0bb7/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:19:08.607724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2286511-509b-49ea-bde1-bfd49028302b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:08.608232 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2286511-509b-49ea-bde1-bfd49028302b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:08.608232 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2286511-509b-49ea-bde1-bfd49028302b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:08.609594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 646/1274] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:19:08.615228 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776ff65-71af-40aa-9651-209ed8d9d2a3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:08.615662 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776ff65-71af-40aa-9651-209ed8d9d2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:08.615792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776ff65-71af-40aa-9651-209ed8d9d2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:08.632578 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6776ff65-71af-40aa-9651-209ed8d9d2a3 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:19:08.633742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6776ff65-71af-40aa-9651-209ed8d9d2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:19:08.697575 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 629/1275] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:19:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:19:08.720202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2286511-509b-49ea-bde1-bfd49028302b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:08.825032 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2286511-509b-49ea-bde1-bfd49028302b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:08.843039 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:08.864290 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5a7d6-98dd-4d87-a5c3-8c95f8d00729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:08.865192 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5a7d6-98dd-4d87-a5c3-8c95f8d00729 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:08.865479 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5a7d6-98dd-4d87-a5c3-8c95f8d00729 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:08.865761 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5a7d6-98dd-4d87-a5c3-8c95f8d00729 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:08.875818 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b5a7d6-98dd-4d87-a5c3-8c95f8d00729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 10:19:08.883358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a3b5a7d6-98dd-4d87-a5c3-8c95f8d00729 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:19:08.883358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:08.883358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:19:08.883358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:19:08.883358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:19:08.883358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:19:08.883358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:19:08.883358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:19:08.883358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:19:08.883358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:19:08.883358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:19:08.883358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:19:08.883358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 10:19:08.883358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 10:19:08.883358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 10:19:08.883358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 10:19:08.884470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 10:19:08.884470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 10:19:08.884470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 10:19:08.884470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 10:19:08.884470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:19:08.884470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:19:08.884470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:19:08.884470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:19:08.884470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:19:08.884470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:19:08.884470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:19:08.884470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:19:08.884470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:19:08.884470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:19:08.884470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:19:08.884470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:19:08.885372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:19:08.885372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:19:08.885372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:19:08.885372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:19:08.885372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 10:19:08.885372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 10:19:08.885372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 10:19:08.885372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 10:19:08.885372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 10:19:08.885372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 10:19:08.885372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 10:19:08.885372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:19:08.885372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 647/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jul 18 10:19:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:19:08.890677 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776ff65-71af-40aa-9651-209ed8d9d2a3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:08.969517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:08.989888 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776ff65-71af-40aa-9651-209ed8d9d2a3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:09.021943 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956bc7b-d8b3-4bd9-8092-72e33b8f04af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:09.022591 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956bc7b-d8b3-4bd9-8092-72e33b8f04af None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:09.022873 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956bc7b-d8b3-4bd9-8092-72e33b8f04af None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:09.023146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956bc7b-d8b3-4bd9-8092-72e33b8f04af None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:09.023883 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956bc7b-d8b3-4bd9-8092-72e33b8f04af None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:09.024186 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956bc7b-d8b3-4bd9-8092-72e33b8f04af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:09.025583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956bc7b-d8b3-4bd9-8092-72e33b8f04af None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:09.033299 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 630/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jul 18 10:19:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:09.038896 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5a7d6-98dd-4d87-a5c3-8c95f8d00729 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:09.136335 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5a7d6-98dd-4d87-a5c3-8c95f8d00729 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:09.169734 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3fb83-02cd-4930-b234-edf4fc26adcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:09.170519 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3fb83-02cd-4930-b234-edf4fc26adcc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:09.170804 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3fb83-02cd-4930-b234-edf4fc26adcc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:09.171080 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3fb83-02cd-4930-b234-edf4fc26adcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:09.181389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a3fb83-02cd-4930-b234-edf4fc26adcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:09.182832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a3fb83-02cd-4930-b234-edf4fc26adcc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:09.199552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-16a3fb83-02cd-4930-b234-edf4fc26adcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:19:09.199552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:09.199552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:19:09.199552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:19:09.199552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:19:09.199552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:19:09.199552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:19:09.199552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:19:09.199552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:19:09.199552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:19:09.199552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:19:09.199552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:19:09.199552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:19:09.199552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:19:09.199552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:19:09.200710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:19:09.200710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:19:09.200710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:19:09.200710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:19:09.200710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:19:09.200710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:19:09.200710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:19:09.200710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:19:09.200710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:19:09.200710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:19:09.200710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:19:09.200710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:19:09.200710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:19:09.200710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:19:09.200710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:19:09.200710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:19:09.200710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:19:09.201995 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:19:09.201995 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:19:09.201995 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:19:09.201995 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:19:09.201995 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 648/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:19:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:19:09.206654 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956bc7b-d8b3-4bd9-8092-72e33b8f04af None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:09.304514 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956bc7b-d8b3-4bd9-8092-72e33b8f04af None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:09.336628 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24435af-37ba-4f39-a639-88a09ab2ce99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:09.337282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24435af-37ba-4f39-a639-88a09ab2ce99 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:09.337572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24435af-37ba-4f39-a639-88a09ab2ce99 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:09.337853 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24435af-37ba-4f39-a639-88a09ab2ce99 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:09.338571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24435af-37ba-4f39-a639-88a09ab2ce99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:09.338884 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24435af-37ba-4f39-a639-88a09ab2ce99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:09.340237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24435af-37ba-4f39-a639-88a09ab2ce99 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:09.352084 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 631/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:09.357949 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3fb83-02cd-4930-b234-edf4fc26adcc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:09.388301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:09.458470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3fb83-02cd-4930-b234-edf4fc26adcc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:09.490460 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64098c7-4af0-469f-84ca-9f2aa28c95f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:09.491237 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64098c7-4af0-469f-84ca-9f2aa28c95f4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:09.491523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64098c7-4af0-469f-84ca-9f2aa28c95f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:09.491801 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64098c7-4af0-469f-84ca-9f2aa28c95f4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:09.498963 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64098c7-4af0-469f-84ca-9f2aa28c95f4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:09.500394 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64098c7-4af0-469f-84ca-9f2aa28c95f4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:09.514837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e64098c7-4af0-469f-84ca-9f2aa28c95f4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:19:09.514837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:09.514837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:19:09.514837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:19:09.514837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:19:09.514837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:19:09.514837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:19:09.514837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:19:09.514837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:19:09.514837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:19:09.514837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:19:09.514837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:19:09.514837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 10:19:09.514837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 10:19:09.514837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 10:19:09.514837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 10:19:09.515798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:19:09.515798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:19:09.515798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 10:19:09.515798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:19:09.515798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 10:19:09.515798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 10:19:09.515798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:19:09.515798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:19:09.515798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:19:09.515798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:19:09.515798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:19:09.515798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:19:09.515798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:19:09.515798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:19:09.515798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:19:09.515798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:19:09.515798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:19:09.515798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:19:09.516867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:19:09.516867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:19:09.516867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 10:19:09.516867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 10:19:09.516867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 10:19:09.516867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 10:19:09.516867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 10:19:09.516867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 10:19:09.516867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 10:19:09.516867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:19:09.516867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 649/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:19:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:19:09.522333 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24435af-37ba-4f39-a639-88a09ab2ce99 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:09.526024 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:09.617549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24435af-37ba-4f39-a639-88a09ab2ce99 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:09.650600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ab737-3dd2-4d66-881b-f9d210b801d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:09.651208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ab737-3dd2-4d66-881b-f9d210b801d5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:09.651496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ab737-3dd2-4d66-881b-f9d210b801d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:09.651778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ab737-3dd2-4d66-881b-f9d210b801d5 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:09.652634 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ab737-3dd2-4d66-881b-f9d210b801d5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:09.652951 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ab737-3dd2-4d66-881b-f9d210b801d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:09.654375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ab737-3dd2-4d66-881b-f9d210b801d5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:09.670654 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 632/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Jul 18 10:19:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:09.676232 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64098c7-4af0-469f-84ca-9f2aa28c95f4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:09.764498 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:09.773921 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64098c7-4af0-469f-84ca-9f2aa28c95f4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:09.809180 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166e12d-7311-40c5-9f3a-2d77c91c1817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:09.809829 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166e12d-7311-40c5-9f3a-2d77c91c1817 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:09.810133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166e12d-7311-40c5-9f3a-2d77c91c1817 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:09.810494 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166e12d-7311-40c5-9f3a-2d77c91c1817 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:09.876868 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166e12d-7311-40c5-9f3a-2d77c91c1817 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:09.878326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166e12d-7311-40c5-9f3a-2d77c91c1817 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:09.963050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:09.973328 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 650/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jul 18 10:19:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 10:19:09.979044 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ab737-3dd2-4d66-881b-f9d210b801d5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:10.040310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:10.078258 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ab737-3dd2-4d66-881b-f9d210b801d5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:10.110553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71d442-c6ab-4419-a0de-18c68eb88ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:10.111329 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71d442-c6ab-4419-a0de-18c68eb88ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:10.111617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71d442-c6ab-4419-a0de-18c68eb88ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:10.111895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71d442-c6ab-4419-a0de-18c68eb88ca1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:10.123064 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71d442-c6ab-4419-a0de-18c68eb88ca1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:10.124530 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71d442-c6ab-4419-a0de-18c68eb88ca1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:10.142039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-af71d442-c6ab-4419-a0de-18c68eb88ca1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:19:10.142039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:10.142039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:19:10.142039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:19:10.142039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:19:10.142039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:19:10.142039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:19:10.142039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:19:10.142039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:19:10.142039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:19:10.142039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:19:10.142039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:19:10.142039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 10:19:10.142039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 10:19:10.142039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 10:19:10.143099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 10:19:10.143099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:19:10.143099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:19:10.143099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:19:10.143099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:19:10.143099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:19:10.143099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:19:10.143099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:19:10.143099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:19:10.143099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:19:10.143099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:19:10.143099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:19:10.143099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:19:10.143099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:19:10.143099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:19:10.143099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:19:10.143099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:19:10.144338 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 10:19:10.144338 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 10:19:10.144338 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 10:19:10.144338 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:19:10.144338 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 633/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:19:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:19:10.148907 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166e12d-7311-40c5-9f3a-2d77c91c1817 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:10.246512 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166e12d-7311-40c5-9f3a-2d77c91c1817 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:10.280922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75b4c3-916c-4cc9-9465-15f7ef0cb53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:10.281581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75b4c3-916c-4cc9-9465-15f7ef0cb53d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:10.281867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75b4c3-916c-4cc9-9465-15f7ef0cb53d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:10.282152 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75b4c3-916c-4cc9-9465-15f7ef0cb53d None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:10.282875 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b75b4c3-916c-4cc9-9465-15f7ef0cb53d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:10.283185 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b75b4c3-916c-4cc9-9465-15f7ef0cb53d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:10.284591 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b75b4c3-916c-4cc9-9465-15f7ef0cb53d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:10.295533 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 651/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jul 18 10:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:10.301168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71d442-c6ab-4419-a0de-18c68eb88ca1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:10.398299 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71d442-c6ab-4419-a0de-18c68eb88ca1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:10.416452 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:10.438092 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff935ad-d32e-4a69-b3fb-7a29b1025430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:10.438880 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff935ad-d32e-4a69-b3fb-7a29b1025430 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:10.439169 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff935ad-d32e-4a69-b3fb-7a29b1025430 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:10.439449 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff935ad-d32e-4a69-b3fb-7a29b1025430 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:10.450643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff935ad-d32e-4a69-b3fb-7a29b1025430 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:10.453085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff935ad-d32e-4a69-b3fb-7a29b1025430 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:10.467779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-dff935ad-d32e-4a69-b3fb-7a29b1025430 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:19:10.467779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:10.467779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:19:10.467779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:19:10.467779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:19:10.467779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:19:10.467779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:19:10.467779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:19:10.467779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:19:10.467779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:19:10.467779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:19:10.467779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:19:10.467779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 10:19:10.467779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 10:19:10.467779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 10:19:10.468894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 10:19:10.468894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:19:10.468894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:19:10.468894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:19:10.468894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:19:10.468894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:19:10.468894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:19:10.468894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:19:10.468894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 18 10:19:10.468894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:19:10.468894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:19:10.468894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:19:10.468894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:19:10.468894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:19:10.468894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:19:10.468894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:19:10.468894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:19:10.470027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 10:19:10.470027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 10:19:10.470027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 10:19:10.470027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 10:19:10.470027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 10:19:10.470027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 10:19:10.470027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 10:19:10.470027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 18 10:19:10.470027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 634/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jul 18 10:19:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:19:10.475219 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75b4c3-916c-4cc9-9465-15f7ef0cb53d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:10.523789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:10.571250 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75b4c3-916c-4cc9-9465-15f7ef0cb53d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:10.602951 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e4d9b-4ad2-4f98-b2db-b2b99698bf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:10.603596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e4d9b-4ad2-4f98-b2db-b2b99698bf91 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:10.603881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e4d9b-4ad2-4f98-b2db-b2b99698bf91 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:10.604159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e4d9b-4ad2-4f98-b2db-b2b99698bf91 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:10.604971 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e4d9b-4ad2-4f98-b2db-b2b99698bf91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:10.605283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e4d9b-4ad2-4f98-b2db-b2b99698bf91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:10.606975 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71e4d9b-4ad2-4f98-b2db-b2b99698bf91 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:10.619038 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 652/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jul 18 10:19:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:10.625004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff935ad-d32e-4a69-b3fb-7a29b1025430 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:10.719985 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff935ad-d32e-4a69-b3fb-7a29b1025430 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:10.751948 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5832f-50ca-4ef6-af24-2d14f27aef94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:10.752600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5832f-50ca-4ef6-af24-2d14f27aef94 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:10.752887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5832f-50ca-4ef6-af24-2d14f27aef94 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:10.753164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5832f-50ca-4ef6-af24-2d14f27aef94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:10.754748 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5832f-50ca-4ef6-af24-2d14f27aef94 None admin] RBAC: Adding query filter params (role.id=4d85c69b727b4fac8d2b15e4446bc949, scope.project.id=e71a3acd0adf46a2a35b965a27c4998f, user.id=d07165f554a445169b2c3f0d173c0bb7) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:10.755205 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5832f-50ca-4ef6-af24-2d14f27aef94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:10.756719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5832f-50ca-4ef6-af24-2d14f27aef94 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:10.762336 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 635/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jul 18 10:19:10 2022] GET /identity/v3/role_assignments?user.id=d07165f554a445169b2c3f0d173c0bb7&scope.project.id=e71a3acd0adf46a2a35b965a27c4998f&role.id=4d85c69b727b4fac8d2b15e4446bc949 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:12.767428 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3def7-9a96-4bfa-837d-24a42b97c3f2 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:12.768008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3def7-9a96-4bfa-837d-24a42b97c3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:12.768008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3def7-9a96-4bfa-837d-24a42b97c3f2 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:12.769372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 653/1288] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:19:12.774379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1829fe-362e-4490-84f3-6482b244787e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:12.774772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1829fe-362e-4490-84f3-6482b244787e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:12.774939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1829fe-362e-4490-84f3-6482b244787e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:12.791470 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0c1829fe-362e-4490-84f3-6482b244787e None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:19:12.792909 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0c1829fe-362e-4490-84f3-6482b244787e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:19:12.828728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:12.872079 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 636/1289] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:19:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:19:12.894536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3def7-9a96-4bfa-837d-24a42b97c3f2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:12.991794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3def7-9a96-4bfa-837d-24a42b97c3f2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:13.001992 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:13.025616 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83c599-e17b-4772-87d2-7e0bdcf0175e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:13.026339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83c599-e17b-4772-87d2-7e0bdcf0175e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:13.026617 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83c599-e17b-4772-87d2-7e0bdcf0175e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:13.026885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83c599-e17b-4772-87d2-7e0bdcf0175e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:13.027524 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83c599-e17b-4772-87d2-7e0bdcf0175e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:13.028910 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83c599-e17b-4772-87d2-7e0bdcf0175e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:13.040309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-bc83c599-e17b-4772-87d2-7e0bdcf0175e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:19:13.040309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:13.040309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:19:13.040309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:19:13.040309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:19:13.040309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:19:13.040309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:19:13.040309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:19:13.040309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:19:13.040309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:19:13.040309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:19:13.040309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:19:13.040309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:13.040309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:19:13.040309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:19:13.040309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:19:13.040309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:19:13.041504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:19:13.041504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:19:13.041504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:19:13.041504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:19:13.041504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:19:13.041504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:19:13.041504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:19:13.041504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:19:13.041504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:19:13.041504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:19:13.041504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:19:13.041504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:19:13.041504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:19:13.041504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:19:13.041504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:19:13.041504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:19:13.042442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:19:13.042442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:19:13.042442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:19:13.042442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:19:13.042442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:19:13.042442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:19:13.042442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:19:13.042442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:19:13.042442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:19:13.042442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:19:13.042442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:19:13.042442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:19:13.042442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:19:13.042442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:19:13.042442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 654/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 18 10:19:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:19:13.047332 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1829fe-362e-4490-84f3-6482b244787e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:13.141565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1829fe-362e-4490-84f3-6482b244787e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:13.174412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8b394-19b4-44d7-ad2f-c702b857e27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:13.175073 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8b394-19b4-44d7-ad2f-c702b857e27a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:13.175371 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8b394-19b4-44d7-ad2f-c702b857e27a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:13.176187 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8b394-19b4-44d7-ad2f-c702b857e27a None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:13.176920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b8b394-19b4-44d7-ad2f-c702b857e27a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:13.177227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b8b394-19b4-44d7-ad2f-c702b857e27a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:13.178590 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b8b394-19b4-44d7-ad2f-c702b857e27a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:13.183376 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 637/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Jul 18 10:19:13 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:13.189080 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83c599-e17b-4772-87d2-7e0bdcf0175e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:13.286680 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83c599-e17b-4772-87d2-7e0bdcf0175e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:13.319183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21eb5d-f8f4-4256-a698-b51ef98ae0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:13.319843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21eb5d-f8f4-4256-a698-b51ef98ae0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:13.320129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21eb5d-f8f4-4256-a698-b51ef98ae0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:13.320407 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21eb5d-f8f4-4256-a698-b51ef98ae0c1 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:13.321155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21eb5d-f8f4-4256-a698-b51ef98ae0c1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:13.321465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21eb5d-f8f4-4256-a698-b51ef98ae0c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:13.322835 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21eb5d-f8f4-4256-a698-b51ef98ae0c1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:13.327360 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 655/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Jul 18 10:19:13 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:15.287710 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065786c-4948-401e-b860-b0aa630a8f4b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:15.288240 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065786c-4948-401e-b860-b0aa630a8f4b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:15.288361 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065786c-4948-401e-b860-b0aa630a8f4b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:15.289684 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 638/1293] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:19:15.294875 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608db5bd-8087-4213-b04c-82e3e64911dd None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:15.295294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608db5bd-8087-4213-b04c-82e3e64911dd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:15.295433 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608db5bd-8087-4213-b04c-82e3e64911dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:15.311962 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-608db5bd-8087-4213-b04c-82e3e64911dd None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:19:15.313164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-608db5bd-8087-4213-b04c-82e3e64911dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:19:15.341637 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:15.381680 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 656/1294] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:19:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:19:15.404492 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065786c-4948-401e-b860-b0aa630a8f4b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:15.435703 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:15.504987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065786c-4948-401e-b860-b0aa630a8f4b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:15.537593 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb8761-09d8-4560-a7da-32a7f7b5bb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:15.538240 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb8761-09d8-4560-a7da-32a7f7b5bb0f None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:15.538525 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb8761-09d8-4560-a7da-32a7f7b5bb0f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:15.538803 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb8761-09d8-4560-a7da-32a7f7b5bb0f None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:15.539528 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb8761-09d8-4560-a7da-32a7f7b5bb0f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:15.540915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb8761-09d8-4560-a7da-32a7f7b5bb0f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:15.553277 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 639/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jul 18 10:19:15 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:19:17.569237 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de057d30-1352-41d6-8284-a1d806ec48cf None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:17.569757 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de057d30-1352-41d6-8284-a1d806ec48cf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:17.569757 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de057d30-1352-41d6-8284-a1d806ec48cf None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:17.571094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 657/1296] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:19:17.576242 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b95a63-0fc1-44f6-bc2a-62aa67133aa2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:17.576692 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b95a63-0fc1-44f6-bc2a-62aa67133aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:17.576834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b95a63-0fc1-44f6-bc2a-62aa67133aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:17.593689 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b3b95a63-0fc1-44f6-bc2a-62aa67133aa2 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:19:17.594880 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b3b95a63-0fc1-44f6-bc2a-62aa67133aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:19:17.665762 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 640/1297] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:19:17 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 10:19:17.687639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de057d30-1352-41d6-8284-a1d806ec48cf None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:17.788104 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de057d30-1352-41d6-8284-a1d806ec48cf None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:17.822831 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd77235-ed39-4992-ae44-fde718d8670d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:17.823560 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd77235-ed39-4992-ae44-fde718d8670d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:17.823840 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd77235-ed39-4992-ae44-fde718d8670d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:17.824114 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd77235-ed39-4992-ae44-fde718d8670d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:17.824800 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd77235-ed39-4992-ae44-fde718d8670d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:17.826191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd77235-ed39-4992-ae44-fde718d8670d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:17.840131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-1dd77235-ed39-4992-ae44-fde718d8670d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:19:17.840131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:17.840131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:19:17.840131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:19:17.840131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:19:17.840131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:19:17.840131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:19:17.840131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:19:17.840131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:19:17.840131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:19:17.840131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:19:17.840131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:19:17.840131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:17.840131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:19:17.840131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:19:17.840131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:19:17.840131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:19:17.841260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:19:17.841260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:19:17.841260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:19:17.841260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:19:17.841260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:19:17.841260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:19:17.841260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:19:17.841260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:19:17.841260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:19:17.841260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:19:17.841260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:19:17.841260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:19:17.841260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:19:17.841260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:19:17.841260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:19:17.841260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:19:17.842111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:19:17.842111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:19:17.842111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:19:17.842111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:19:17.842111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:19:17.842111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:19:17.842111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:19:17.842111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:19:17.842111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:19:17.842111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:19:17.842111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:19:17.842111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:19:17.842111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:19:17.842111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:19:17.842111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 658/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 18 10:19:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:19:17.847955 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b95a63-0fc1-44f6-bc2a-62aa67133aa2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:17.866458 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:17.944429 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b95a63-0fc1-44f6-bc2a-62aa67133aa2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:17.977261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cd670-cd35-4e2e-8348-59b3c3873d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:17.977906 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cd670-cd35-4e2e-8348-59b3c3873d06 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:17.978188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cd670-cd35-4e2e-8348-59b3c3873d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:17.978463 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cd670-cd35-4e2e-8348-59b3c3873d06 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:17.979162 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1cd670-cd35-4e2e-8348-59b3c3873d06 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:17.979468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1cd670-cd35-4e2e-8348-59b3c3873d06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:17.980851 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1cd670-cd35-4e2e-8348-59b3c3873d06 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:17.986057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 641/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Jul 18 10:19:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:19:17.992475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd77235-ed39-4992-ae44-fde718d8670d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:18.023100 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:18.088426 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd77235-ed39-4992-ae44-fde718d8670d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:18.120599 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd413f1-5f4b-4486-bf5e-46d297cfb1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:18.121238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd413f1-5f4b-4486-bf5e-46d297cfb1bc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:18.121523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd413f1-5f4b-4486-bf5e-46d297cfb1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:18.121799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd413f1-5f4b-4486-bf5e-46d297cfb1bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:18.122579 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd413f1-5f4b-4486-bf5e-46d297cfb1bc None admin] RBAC: Adding query filter params (interface=public, service_id=e66fcdc213ac49d58035d32aee0d6b9d, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:18.122887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd413f1-5f4b-4486-bf5e-46d297cfb1bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:18.124240 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd413f1-5f4b-4486-bf5e-46d297cfb1bc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:18.129620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 659/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jul 18 10:19:17 2022] GET /identity/v3/endpoints?service_id=e66fcdc213ac49d58035d32aee0d6b9d&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:18.135179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cd670-cd35-4e2e-8348-59b3c3873d06 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:18.215469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:18.233190 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cd670-cd35-4e2e-8348-59b3c3873d06 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:18.266500 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4cafe-0007-40a9-aef9-cadb0aa3b6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:18.267153 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4cafe-0007-40a9-aef9-cadb0aa3b6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:18.267441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4cafe-0007-40a9-aef9-cadb0aa3b6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:18.267722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4cafe-0007-40a9-aef9-cadb0aa3b6d7 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:18.268405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4cafe-0007-40a9-aef9-cadb0aa3b6d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:18.270431 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4cafe-0007-40a9-aef9-cadb0aa3b6d7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:18.277119 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 642/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jul 18 10:19:18 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:19:20.276822 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d357e-a498-4057-aab2-b53f958561ce None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:20.277352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d357e-a498-4057-aab2-b53f958561ce None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:20.277452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d357e-a498-4057-aab2-b53f958561ce None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:20.278762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 660/1302] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:19:20.283752 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16c6d9-4eb7-436a-800d-d3c779abbba4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:20.284187 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16c6d9-4eb7-436a-800d-d3c779abbba4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:20.284293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16c6d9-4eb7-436a-800d-d3c779abbba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:20.301102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4e16c6d9-4eb7-436a-800d-d3c779abbba4 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:19:20.302263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4e16c6d9-4eb7-436a-800d-d3c779abbba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:19:20.365379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 643/1303] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:19:20 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 10:19:20.387202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d357e-a498-4057-aab2-b53f958561ce None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:20.480754 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:20.488615 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d357e-a498-4057-aab2-b53f958561ce None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:20.522398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b77e35-9a17-421a-9cea-6878503cf4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:20.523499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b77e35-9a17-421a-9cea-6878503cf4bd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:20.523786 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b77e35-9a17-421a-9cea-6878503cf4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:20.524062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b77e35-9a17-421a-9cea-6878503cf4bd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:20.524746 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b77e35-9a17-421a-9cea-6878503cf4bd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:20.526850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b77e35-9a17-421a-9cea-6878503cf4bd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:20.539497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-00b77e35-9a17-421a-9cea-6878503cf4bd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:19:20.539497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:20.539497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 10:19:20.539497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 10:19:20.539497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 10:19:20.539497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 10:19:20.539497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 10:19:20.539497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:19:20.539497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:19:20.539497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:19:20.539497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 10:19:20.539497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:19:20.539497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 10:19:20.539497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 10:19:20.539497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 10:19:20.539497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 10:19:20.539497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 10:19:20.540771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 10:19:20.540771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 10:19:20.540771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 10:19:20.540771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 10:19:20.540771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 10:19:20.540771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 10:19:20.540771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 10:19:20.540771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 10:19:20.540771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 10:19:20.540771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 10:19:20.540771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 10:19:20.540771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 10:19:20.540771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 10:19:20.540771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 10:19:20.540771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 10:19:20.540771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 10:19:20.541635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 10:19:20.541635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Jul 18 10:19:20.541635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 10:19:20.541635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Jul 18 10:19:20.541635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 10:19:20.541635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 10:19:20.541635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 10:19:20.541635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Jul 18 10:19:20.541635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 10:19:20.541635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Jul 18 10:19:20.541635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 10:19:20.541635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 10:19:20.541635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 10:19:20.541635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Jul 18 10:19:20.541635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 661/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jul 18 10:19:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 10:19:20.546716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16c6d9-4eb7-436a-800d-d3c779abbba4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:20.619333 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:20.642685 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16c6d9-4eb7-436a-800d-d3c779abbba4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:20.675764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3050cc3-972d-4f1d-ad27-924a995fd1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:20.676409 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3050cc3-972d-4f1d-ad27-924a995fd1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:20.676718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3050cc3-972d-4f1d-ad27-924a995fd1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:20.676997 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3050cc3-972d-4f1d-ad27-924a995fd1a6 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:20.677693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3050cc3-972d-4f1d-ad27-924a995fd1a6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:19:20.677997 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3050cc3-972d-4f1d-ad27-924a995fd1a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:20.679348 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3050cc3-972d-4f1d-ad27-924a995fd1a6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:20.684424 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 644/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Jul 18 10:19:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:19:20.690066 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b77e35-9a17-421a-9cea-6878503cf4bd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:19:20.786369 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b77e35-9a17-421a-9cea-6878503cf4bd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:19:20.823307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80948e-8c90-4e3e-84c1-f319a665ed1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:19:20.824265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80948e-8c90-4e3e-84c1-f319a665ed1f None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:19:20.824426 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80948e-8c90-4e3e-84c1-f319a665ed1f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:19:20.824792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80948e-8c90-4e3e-84c1-f319a665ed1f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:19:20.825626 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80948e-8c90-4e3e-84c1-f319a665ed1f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:19:20.827263 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80948e-8c90-4e3e-84c1-f319a665ed1f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:19:20.846267 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 662/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jul 18 10:19:20 2022] POST /identity/v3/endpoints => generated 331 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:19:30.681414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=30 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:19:30.841904 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:23.003674 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82825f03-ce07-4b16-80e6-d0fd50bab008 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:23.004763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82825f03-ce07-4b16-80e6-d0fd50bab008 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:23.005278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82825f03-ce07-4b16-80e6-d0fd50bab008 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:23.007203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 645/1307] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:20:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:20:23.012283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cd7cc-4960-42bb-ad15-6d308cb2db0a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:23.012790 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cd7cc-4960-42bb-ad15-6d308cb2db0a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:23.012887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cd7cc-4960-42bb-ad15-6d308cb2db0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:23.089975 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bd8cd7cc-4960-42bb-ad15-6d308cb2db0a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:20:23.091231 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bd8cd7cc-4960-42bb-ad15-6d308cb2db0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:23.253077 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 663/1308] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:20:23.286395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f4e46-9cae-48b6-8744-f439d0d2dce7 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:23.286760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f4e46-9cae-48b6-8744-f439d0d2dce7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:23.287113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f4e46-9cae-48b6-8744-f439d0d2dce7 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:23.288943 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 646/1309] 10.222.0.33 () {54 vars in 965 bytes} [Mon Jul 18 10:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:20:23.298490 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298b910-3566-455c-b0d6-ed83c4eb5f89 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:23.298791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298b910-3566-455c-b0d6-ed83c4eb5f89 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:23.299105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298b910-3566-455c-b0d6-ed83c4eb5f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:23.391791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1298b910-3566-455c-b0d6-ed83c4eb5f89 None None] MFA Rules not processed for user `10d29b7934374f34923587eda9be0ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:20:23.393682 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1298b910-3566-455c-b0d6-ed83c4eb5f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:23.417250 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:23.478638 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 664/1310] 10.222.0.33 () {58 vars in 1062 bytes} [Mon Jul 18 10:20:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:20:23.488464 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f4e46-9cae-48b6-8744-f439d0d2dce7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:20:23.490462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9d0f4e46-9cae-48b6-8744-f439d0d2dce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:23.618358 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f4e46-9cae-48b6-8744-f439d0d2dce7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:20:23.697867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072365d-c212-480d-9971-b08624d66f83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:20:23.698519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072365d-c212-480d-9971-b08624d66f83 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:23.698805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072365d-c212-480d-9971-b08624d66f83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:23.699081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072365d-c212-480d-9971-b08624d66f83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:23.700749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f072365d-c212-480d-9971-b08624d66f83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:23.757552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:23.816311 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072365d-c212-480d-9971-b08624d66f83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:20:23.817721 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072365d-c212-480d-9971-b08624d66f83 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:20:23.837934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 647/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:20:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:20:26.142375 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e98ac8-5e23-4136-bec5-098694f36e2b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:26.142914 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e98ac8-5e23-4136-bec5-098694f36e2b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:26.143005 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e98ac8-5e23-4136-bec5-098694f36e2b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:26.144361 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 665/1312] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:20:26.149568 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397ce26-bf0b-464b-a847-501d4940b0d0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:26.149968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397ce26-bf0b-464b-a847-501d4940b0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:26.150263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397ce26-bf0b-464b-a847-501d4940b0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:26.236777 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9397ce26-bf0b-464b-a847-501d4940b0d0 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:20:26.238010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9397ce26-bf0b-464b-a847-501d4940b0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:26.348680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 648/1313] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:20:26.385879 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e98ac8-5e23-4136-bec5-098694f36e2b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:20:26.387539 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-09e98ac8-5e23-4136-bec5-098694f36e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:26.498608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e98ac8-5e23-4136-bec5-098694f36e2b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:20:26.573831 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:26.576680 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c0f8f-4e1e-45ef-b32a-171eb79197f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:20:26.577406 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c0f8f-4e1e-45ef-b32a-171eb79197f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:26.578027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c0f8f-4e1e-45ef-b32a-171eb79197f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:26.578630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c0f8f-4e1e-45ef-b32a-171eb79197f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:26.581314 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-293c0f8f-4e1e-45ef-b32a-171eb79197f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:26.703249 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c0f8f-4e1e-45ef-b32a-171eb79197f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:20:26.704720 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c0f8f-4e1e-45ef-b32a-171eb79197f4 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:20:26.722763 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 666/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:20:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:20:29.037538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4f179-d2c5-4b59-8a16-fb55ad5cebb7 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:29.038103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4f179-d2c5-4b59-8a16-fb55ad5cebb7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:29.038103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4f179-d2c5-4b59-8a16-fb55ad5cebb7 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:29.039464 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 649/1315] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:20:29.044385 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a797fb-757f-4b46-8824-d5213c58c137 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:29.044838 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a797fb-757f-4b46-8824-d5213c58c137 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:29.044959 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a797fb-757f-4b46-8824-d5213c58c137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:29.108026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c3a797fb-757f-4b46-8824-d5213c58c137 None None] MFA Rules not processed for user `4eecd6829d5a4a319fa4af969eb3d39d`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:20:29.109244 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c3a797fb-757f-4b46-8824-d5213c58c137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:29.159923 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:29.204527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 667/1316] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:20:29.261252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251bcc7-03f8-4daa-b5f8-180e04b5bc83 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:29.261545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251bcc7-03f8-4daa-b5f8-180e04b5bc83 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:29.261826 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251bcc7-03f8-4daa-b5f8-180e04b5bc83 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:29.263212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 650/1317] 10.222.0.33 () {54 vars in 965 bytes} [Mon Jul 18 10:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:20:29.270182 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25146463-2864-43cb-965b-7a7b61d39a90 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:29.270480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25146463-2864-43cb-965b-7a7b61d39a90 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:29.270762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25146463-2864-43cb-965b-7a7b61d39a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:29.351878 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-25146463-2864-43cb-965b-7a7b61d39a90 None None] MFA Rules not processed for user `10d29b7934374f34923587eda9be0ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:20:29.353032 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-25146463-2864-43cb-965b-7a7b61d39a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:29.426781 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 668/1318] 10.222.0.33 () {58 vars in 1062 bytes} [Mon Jul 18 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:20:29.435770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251bcc7-03f8-4daa-b5f8-180e04b5bc83 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:20:29.437339 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9251bcc7-03f8-4daa-b5f8-180e04b5bc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:29.446894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:29.541322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251bcc7-03f8-4daa-b5f8-180e04b5bc83 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:20:29.613727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc6d10-3eb7-4432-aad3-610fe2a1e27c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:20:29.615170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc6d10-3eb7-4432-aad3-610fe2a1e27c service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:29.615170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc6d10-3eb7-4432-aad3-610fe2a1e27c service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:29.615170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc6d10-3eb7-4432-aad3-610fe2a1e27c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:29.616532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6fdc6d10-3eb7-4432-aad3-610fe2a1e27c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:29.677848 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:29.729369 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc6d10-3eb7-4432-aad3-610fe2a1e27c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:20:29.730817 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc6d10-3eb7-4432-aad3-610fe2a1e27c service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:20:29.749758 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 651/1319] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:20:29 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:20:32.637247 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159c090-212e-4a83-b396-df576befac72 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:32.637700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159c090-212e-4a83-b396-df576befac72 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:32.637795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159c090-212e-4a83-b396-df576befac72 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:32.639194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 669/1320] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:20:32.644454 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fbad9-77d0-4135-a1db-a6360488dfbd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:32.644895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fbad9-77d0-4135-a1db-a6360488dfbd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:32.645041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fbad9-77d0-4135-a1db-a6360488dfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:32.707341 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-717fbad9-77d0-4135-a1db-a6360488dfbd None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:20:32.708549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-717fbad9-77d0-4135-a1db-a6360488dfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:32.803829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 652/1321] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:20:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:20:32.844002 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159c090-212e-4a83-b396-df576befac72 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:20:32.845568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a159c090-212e-4a83-b396-df576befac72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:32.895258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:32.948211 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159c090-212e-4a83-b396-df576befac72 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:20:33.020461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd2e533-2786-4d4a-857f-57cfe0d73fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:20:33.021124 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2e533-2786-4d4a-857f-57cfe0d73fbd service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:33.021407 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2e533-2786-4d4a-857f-57cfe0d73fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:33.021679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2e533-2786-4d4a-857f-57cfe0d73fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:33.023309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1dd2e533-2786-4d4a-857f-57cfe0d73fbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:33.116150 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:33.129631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd2e533-2786-4d4a-857f-57cfe0d73fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:20:33.131088 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd2e533-2786-4d4a-857f-57cfe0d73fbd service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:20:33.148979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 670/1322] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:20:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:20:35.831002 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb7528-0ff5-41dd-9bf2-7d20f03bcba5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:35.831561 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb7528-0ff5-41dd-9bf2-7d20f03bcba5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:35.831561 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb7528-0ff5-41dd-9bf2-7d20f03bcba5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:35.832971 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 653/1323] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:20:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:20:35.838188 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43e315-5d68-4054-8ca5-e8c61672a115 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:35.838597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43e315-5d68-4054-8ca5-e8c61672a115 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:35.838765 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43e315-5d68-4054-8ca5-e8c61672a115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:35.903118 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9d43e315-5d68-4054-8ca5-e8c61672a115 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:20:35.904312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9d43e315-5d68-4054-8ca5-e8c61672a115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:36.004558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 671/1324] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:20:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:20:36.040815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb7528-0ff5-41dd-9bf2-7d20f03bcba5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:20:36.042411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-59fb7528-0ff5-41dd-9bf2-7d20f03bcba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:36.062850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:36.151988 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb7528-0ff5-41dd-9bf2-7d20f03bcba5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:20:36.228095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b38da-a8da-4e22-b336-c958b8ce2102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:20:36.228778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b38da-a8da-4e22-b336-c958b8ce2102 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:36.229063 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b38da-a8da-4e22-b336-c958b8ce2102 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:36.229338 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b38da-a8da-4e22-b336-c958b8ce2102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:36.230976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-030b38da-a8da-4e22-b336-c958b8ce2102 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:36.301607 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:36.339999 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b38da-a8da-4e22-b336-c958b8ce2102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:20:36.341486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b38da-a8da-4e22-b336-c958b8ce2102 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:20:36.360777 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 654/1325] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:20:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:20:38.419406 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8b18c-9a89-4007-9034-e8467fe21abf None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:38.419929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8b18c-9a89-4007-9034-e8467fe21abf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:38.420035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8b18c-9a89-4007-9034-e8467fe21abf None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:38.421437 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 672/1326] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:20:38.426650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded8de6-b738-4ac2-9653-0f7700f1b684 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:38.427073 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded8de6-b738-4ac2-9653-0f7700f1b684 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:38.427190 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded8de6-b738-4ac2-9653-0f7700f1b684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:38.490357 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bded8de6-b738-4ac2-9653-0f7700f1b684 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:20:38.491480 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bded8de6-b738-4ac2-9653-0f7700f1b684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:38.592127 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 655/1327] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:20:38.629307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8b18c-9a89-4007-9034-e8467fe21abf None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:20:38.630857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fbb8b18c-9a89-4007-9034-e8467fe21abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:38.691181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:38.735054 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8b18c-9a89-4007-9034-e8467fe21abf None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:20:38.811246 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c33ec8-c891-4612-a0d1-36b0412a4bfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:20:38.811900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c33ec8-c891-4612-a0d1-36b0412a4bfb service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:38.812187 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c33ec8-c891-4612-a0d1-36b0412a4bfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:38.812466 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c33ec8-c891-4612-a0d1-36b0412a4bfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:38.814157 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-73c33ec8-c891-4612-a0d1-36b0412a4bfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:38.926551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:38.929011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c33ec8-c891-4612-a0d1-36b0412a4bfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:20:38.930761 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c33ec8-c891-4612-a0d1-36b0412a4bfb service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:20:38.949505 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 673/1328] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:20:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:20:41.003067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990380d-150b-4f41-b383-1e3944f933f3 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:41.003619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990380d-150b-4f41-b383-1e3944f933f3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:41.003619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990380d-150b-4f41-b383-1e3944f933f3 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:41.005210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 656/1329] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:20:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:20:41.010952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944b07c-5f72-43cd-9cf7-73f05954a3d0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:41.011466 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944b07c-5f72-43cd-9cf7-73f05954a3d0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:41.011637 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944b07c-5f72-43cd-9cf7-73f05954a3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:41.076175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1944b07c-5f72-43cd-9cf7-73f05954a3d0 None None] MFA Rules not processed for user `4eecd6829d5a4a319fa4af969eb3d39d`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:20:41.077438 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1944b07c-5f72-43cd-9cf7-73f05954a3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:41.175991 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 674/1330] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:20:41 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:20:41.214848 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990380d-150b-4f41-b383-1e3944f933f3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:20:41.216369 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0990380d-150b-4f41-b383-1e3944f933f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:41.247931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:41.325192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990380d-150b-4f41-b383-1e3944f933f3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:20:41.410954 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f9f1d-8c86-40b5-80cc-bdb987ecd3b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:20:41.411608 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f9f1d-8c86-40b5-80cc-bdb987ecd3b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:41.411889 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f9f1d-8c86-40b5-80cc-bdb987ecd3b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:41.412167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f9f1d-8c86-40b5-80cc-bdb987ecd3b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:41.413877 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2f8f9f1d-8c86-40b5-80cc-bdb987ecd3b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:41.491846 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:41.524116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f9f1d-8c86-40b5-80cc-bdb987ecd3b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:20:41.525591 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f9f1d-8c86-40b5-80cc-bdb987ecd3b5 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:20:41.543914 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 657/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:20:41 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:20:43.849241 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed6c1e-fcc9-4315-909f-bfce0c2cdb3c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:43.849888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed6c1e-fcc9-4315-909f-bfce0c2cdb3c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:43.849888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed6c1e-fcc9-4315-909f-bfce0c2cdb3c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:43.851223 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 675/1332] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:20:43.856962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be577f6a-d29b-459d-ac30-ea05a11f3813 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:43.857403 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be577f6a-d29b-459d-ac30-ea05a11f3813 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:43.857599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be577f6a-d29b-459d-ac30-ea05a11f3813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:43.922299 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-be577f6a-d29b-459d-ac30-ea05a11f3813 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:20:43.924081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-be577f6a-d29b-459d-ac30-ea05a11f3813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:44.026997 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 658/1333] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:20:44.063999 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed6c1e-fcc9-4315-909f-bfce0c2cdb3c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:20:44.065576 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-63ed6c1e-fcc9-4315-909f-bfce0c2cdb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:44.134676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:44.170161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed6c1e-fcc9-4315-909f-bfce0c2cdb3c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:20:44.245700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4dfc0b-32b6-4b5f-9848-1849fce4d8e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:20:44.246345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dfc0b-32b6-4b5f-9848-1849fce4d8e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:44.246628 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dfc0b-32b6-4b5f-9848-1849fce4d8e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:44.246903 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dfc0b-32b6-4b5f-9848-1849fce4d8e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:44.248576 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bc4dfc0b-32b6-4b5f-9848-1849fce4d8e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:44.356935 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4dfc0b-32b6-4b5f-9848-1849fce4d8e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:20:44.358373 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4dfc0b-32b6-4b5f-9848-1849fce4d8e0 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:20:44.368596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:44.377548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 676/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:20:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:20:46.473897 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5a4b6-764a-49a9-a5ca-f26775936424 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:46.474417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5a4b6-764a-49a9-a5ca-f26775936424 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:46.474417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5a4b6-764a-49a9-a5ca-f26775936424 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:46.475759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 659/1335] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:20:46.480967 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a388648-4278-4d26-8901-c36f7fccca8b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:46.481649 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a388648-4278-4d26-8901-c36f7fccca8b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:46.481783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a388648-4278-4d26-8901-c36f7fccca8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:46.546127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8a388648-4278-4d26-8901-c36f7fccca8b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:20:46.547323 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8a388648-4278-4d26-8901-c36f7fccca8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:46.643501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 677/1336] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:20:46.683757 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5a4b6-764a-49a9-a5ca-f26775936424 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:20:46.685377 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e3b5a4b6-764a-49a9-a5ca-f26775936424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:46.724853 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:46.788887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5a4b6-764a-49a9-a5ca-f26775936424 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:20:46.861905 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2e245-4b2f-4040-88df-1d0dc26c2b5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:20:46.862554 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2e245-4b2f-4040-88df-1d0dc26c2b5f service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:46.862838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2e245-4b2f-4040-88df-1d0dc26c2b5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:46.863114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2e245-4b2f-4040-88df-1d0dc26c2b5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:46.864789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-beb2e245-4b2f-4040-88df-1d0dc26c2b5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:46.946588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:46.972398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2e245-4b2f-4040-88df-1d0dc26c2b5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:20:46.973866 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2e245-4b2f-4040-88df-1d0dc26c2b5f service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:20:46.993030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 660/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:20:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:20:49.459746 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fb6e2-1d10-4d0b-bcf9-759a3bc355e5 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:49.460293 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fb6e2-1d10-4d0b-bcf9-759a3bc355e5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:49.460293 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fb6e2-1d10-4d0b-bcf9-759a3bc355e5 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:49.461652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 678/1338] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:20:49.466693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89909fc-8564-4fa1-9312-3fcfe2631683 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:49.467092 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89909fc-8564-4fa1-9312-3fcfe2631683 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:49.467246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89909fc-8564-4fa1-9312-3fcfe2631683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:49.535071 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a89909fc-8564-4fa1-9312-3fcfe2631683 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:20:49.536303 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a89909fc-8564-4fa1-9312-3fcfe2631683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:49.630030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 661/1339] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:20:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:20:49.666476 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fb6e2-1d10-4d0b-bcf9-759a3bc355e5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:20:49.668016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-248fb6e2-1d10-4d0b-bcf9-759a3bc355e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:49.743135 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:49.771420 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fb6e2-1d10-4d0b-bcf9-759a3bc355e5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:20:49.849388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7b396-2fd4-42c1-b361-08cf306842a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:20:49.850035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7b396-2fd4-42c1-b361-08cf306842a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:49.850327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7b396-2fd4-42c1-b361-08cf306842a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:49.850608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7b396-2fd4-42c1-b361-08cf306842a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:49.852263 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-54a7b396-2fd4-42c1-b361-08cf306842a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:49.960468 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7b396-2fd4-42c1-b361-08cf306842a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:20:49.961948 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7b396-2fd4-42c1-b361-08cf306842a0 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:20:49.980135 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 679/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:20:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:20:53.870364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96d730-4962-425e-98f9-22ab089d9d7e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:53.870877 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96d730-4962-425e-98f9-22ab089d9d7e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:53.870991 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96d730-4962-425e-98f9-22ab089d9d7e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:53.872366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 662/1341] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:20:53.877738 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14de64-4483-4d92-be7f-c72f67cb3d48 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:53.878166 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14de64-4483-4d92-be7f-c72f67cb3d48 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:53.878349 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14de64-4483-4d92-be7f-c72f67cb3d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:53.882692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:53.947171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2d14de64-4483-4d92-be7f-c72f67cb3d48 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:20:53.948404 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2d14de64-4483-4d92-be7f-c72f67cb3d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:54.047941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 680/1342] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:20:54.086861 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96d730-4962-425e-98f9-22ab089d9d7e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:20:54.088401 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7f96d730-4962-425e-98f9-22ab089d9d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:54.136793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:54.194031 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96d730-4962-425e-98f9-22ab089d9d7e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:20:54.266524 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990d400-5e50-495b-925f-8a5f324c74f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:20:54.267731 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990d400-5e50-495b-925f-8a5f324c74f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:54.268014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990d400-5e50-495b-925f-8a5f324c74f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:54.268291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990d400-5e50-495b-925f-8a5f324c74f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:54.269965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0990d400-5e50-495b-925f-8a5f324c74f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:54.362551 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:54.379883 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990d400-5e50-495b-925f-8a5f324c74f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:20:54.381377 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990d400-5e50-495b-925f-8a5f324c74f4 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:20:54.400562 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 663/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:20:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:20:57.014017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24fc34-92d4-4526-8030-0819547f72e4 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:57.014545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24fc34-92d4-4526-8030-0819547f72e4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:57.014545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24fc34-92d4-4526-8030-0819547f72e4 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:57.015896 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 681/1344] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:20:57.021400 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7a21a-092b-496d-bfac-c8210fd9f549 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:57.021916 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7a21a-092b-496d-bfac-c8210fd9f549 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:57.022170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7a21a-092b-496d-bfac-c8210fd9f549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:57.095071 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b2e7a21a-092b-496d-bfac-c8210fd9f549 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:20:57.096317 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b2e7a21a-092b-496d-bfac-c8210fd9f549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:57.199681 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 664/1345] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:20:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:20:57.238073 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24fc34-92d4-4526-8030-0819547f72e4 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:20:57.239644 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ff24fc34-92d4-4526-8030-0819547f72e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:57.326922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:20:57.344810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24fc34-92d4-4526-8030-0819547f72e4 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:20:57.418329 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b575c92-690b-4e3c-a691-ec606359d551 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:20:57.418979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b575c92-690b-4e3c-a691-ec606359d551 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:20:57.419261 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b575c92-690b-4e3c-a691-ec606359d551 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:20:57.419530 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b575c92-690b-4e3c-a691-ec606359d551 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:20:57.421197 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4b575c92-690b-4e3c-a691-ec606359d551 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:20:57.533155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b575c92-690b-4e3c-a691-ec606359d551 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:20:57.534601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b575c92-690b-4e3c-a691-ec606359d551 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:20:57.552254 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 682/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:20:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:21:01.914040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bbd99-234b-48f0-9ab4-a025db86d001 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:01.914545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bbd99-234b-48f0-9ab4-a025db86d001 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:01.914644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bbd99-234b-48f0-9ab4-a025db86d001 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:01.916006 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 665/1347] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:01.921535 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909031c8-e750-4245-937c-324710a6e41e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:01.922011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909031c8-e750-4245-937c-324710a6e41e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:01.922138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909031c8-e750-4245-937c-324710a6e41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:01.937839 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:01.997163 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-909031c8-e750-4245-937c-324710a6e41e None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:01.998403 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-909031c8-e750-4245-937c-324710a6e41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:02.100424 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 683/1348] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:02.139079 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8bbd99-234b-48f0-9ab4-a025db86d001 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:02.140752 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ea8bbd99-234b-48f0-9ab4-a025db86d001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:02.202393 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:02.255717 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8bbd99-234b-48f0-9ab4-a025db86d001 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:02.330706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd027f3f-41c1-4163-a701-fe0ff765543a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:02.331367 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd027f3f-41c1-4163-a701-fe0ff765543a service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:02.331653 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd027f3f-41c1-4163-a701-fe0ff765543a service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:02.331932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd027f3f-41c1-4163-a701-fe0ff765543a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:02.333628 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dd027f3f-41c1-4163-a701-fe0ff765543a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:02.440185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:02.444270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd027f3f-41c1-4163-a701-fe0ff765543a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:02.445740 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd027f3f-41c1-4163-a701-fe0ff765543a service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:02.464484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 666/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:21:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:21:07.324490 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911154b-0f3c-4f7c-a0c2-0bd3adef2b3e None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:07.325008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911154b-0f3c-4f7c-a0c2-0bd3adef2b3e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:07.325107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911154b-0f3c-4f7c-a0c2-0bd3adef2b3e None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:07.326646 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 684/1350] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 18 10:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:07.332390 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d16b4-90bc-462b-90d1-06efaeeb9a32 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:07.332879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d16b4-90bc-462b-90d1-06efaeeb9a32 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:07.333262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d16b4-90bc-462b-90d1-06efaeeb9a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:07.437333 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4e8d16b4-90bc-462b-90d1-06efaeeb9a32 None None] MFA Rules not processed for user `d07165f554a445169b2c3f0d173c0bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:07.439006 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4e8d16b4-90bc-462b-90d1-06efaeeb9a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:07.602855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 667/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 18 10:21:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:11.731850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc5282-fc63-4d21-b8a4-30443360c8e2 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:11.732341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc5282-fc63-4d21-b8a4-30443360c8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:11.732341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc5282-fc63-4d21-b8a4-30443360c8e2 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:11.733910 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 685/1352] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 18 10:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:11.738854 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd90521-62ff-495e-9ed9-0bca16534f4a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:11.739669 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd90521-62ff-495e-9ed9-0bca16534f4a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:11.740213 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd90521-62ff-495e-9ed9-0bca16534f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:11.807339 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:11.834095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ebd90521-62ff-495e-9ed9-0bca16534f4a None None] MFA Rules not processed for user `d07165f554a445169b2c3f0d173c0bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:11.835697 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ebd90521-62ff-495e-9ed9-0bca16534f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:11.942441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 668/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 18 10:21:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:12.098411 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=59 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:13.422981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8acdf-2260-4cff-ade1-f661350cc301 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:13.423933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8acdf-2260-4cff-ade1-f661350cc301 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:13.423933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8acdf-2260-4cff-ade1-f661350cc301 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:13.425936 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 686/1354] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 18 10:21:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:13.430984 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210d278-e9f3-44e0-89ba-3347d1edda61 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:13.431416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210d278-e9f3-44e0-89ba-3347d1edda61 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:13.431530 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210d278-e9f3-44e0-89ba-3347d1edda61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:13.506476 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c210d278-e9f3-44e0-89ba-3347d1edda61 None None] MFA Rules not processed for user `d07165f554a445169b2c3f0d173c0bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:13.507671 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c210d278-e9f3-44e0-89ba-3347d1edda61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:13.552306 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:13.618001 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 669/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 18 10:21:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:21:16.954769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db520e-a840-4a8e-bc06-cf121b1f6cc0 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:16.955324 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db520e-a840-4a8e-bc06-cf121b1f6cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:16.955324 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db520e-a840-4a8e-bc06-cf121b1f6cc0 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:16.957882 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 687/1356] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:21:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:16.965874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97796c3f-7f16-4e1a-88a7-f0ad4a72b167 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:16.966727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97796c3f-7f16-4e1a-88a7-f0ad4a72b167 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:16.967170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97796c3f-7f16-4e1a-88a7-f0ad4a72b167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:17.054963 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-97796c3f-7f16-4e1a-88a7-f0ad4a72b167 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:17.056174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-97796c3f-7f16-4e1a-88a7-f0ad4a72b167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:17.168602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 670/1357] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:17.220211 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824316fb-9abe-4419-b888-cfda9ee4e8e5 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:17.220563 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824316fb-9abe-4419-b888-cfda9ee4e8e5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:17.220908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824316fb-9abe-4419-b888-cfda9ee4e8e5 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:17.222243 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 688/1358] 10.222.0.33 () {54 vars in 962 bytes} [Mon Jul 18 10:21:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:17.231265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6843f7-0629-45d6-b9c2-a220fa68fe2b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:17.231873 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6843f7-0629-45d6-b9c2-a220fa68fe2b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:17.232463 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6843f7-0629-45d6-b9c2-a220fa68fe2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:17.251317 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:17.314912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3f6843f7-0629-45d6-b9c2-a220fa68fe2b None None] MFA Rules not processed for user `29048c2021e340b081ae44a390583e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:17.316276 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3f6843f7-0629-45d6-b9c2-a220fa68fe2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:17.438762 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 671/1359] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Jul 18 10:21:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:17.447108 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824316fb-9abe-4419-b888-cfda9ee4e8e5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:17.448997 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-824316fb-9abe-4419-b888-cfda9ee4e8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:17.609803 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824316fb-9abe-4419-b888-cfda9ee4e8e5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:17.690773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:17.735228 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255cb98-5274-4e17-83c9-a12fdef539b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:17.737101 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255cb98-5274-4e17-83c9-a12fdef539b7 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:17.737482 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255cb98-5274-4e17-83c9-a12fdef539b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:17.737857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255cb98-5274-4e17-83c9-a12fdef539b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:17.740345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6255cb98-5274-4e17-83c9-a12fdef539b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:17.838522 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255cb98-5274-4e17-83c9-a12fdef539b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:17.840256 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255cb98-5274-4e17-83c9-a12fdef539b7 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:17.859282 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 689/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:21:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:21:20.105117 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a9642-94ef-4c64-8d35-76fd7eede88e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:20.105596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a9642-94ef-4c64-8d35-76fd7eede88e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:20.105683 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a9642-94ef-4c64-8d35-76fd7eede88e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:20.107450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 672/1361] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:20.113033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e804f17-7311-4ae6-abc8-dcc654b9513c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:20.113850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e804f17-7311-4ae6-abc8-dcc654b9513c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:20.114133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e804f17-7311-4ae6-abc8-dcc654b9513c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:20.186318 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8e804f17-7311-4ae6-abc8-dcc654b9513c None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:20.187537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8e804f17-7311-4ae6-abc8-dcc654b9513c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:20.227111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:20.301437 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 690/1362] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:20.346083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a9642-94ef-4c64-8d35-76fd7eede88e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:20.349584 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-705a9642-94ef-4c64-8d35-76fd7eede88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:20.431987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:20.524775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a9642-94ef-4c64-8d35-76fd7eede88e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:20.661162 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb20915-c474-4b53-bbf6-8921003182a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:20.661918 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb20915-c474-4b53-bbf6-8921003182a6 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:20.662205 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb20915-c474-4b53-bbf6-8921003182a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:20.662511 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb20915-c474-4b53-bbf6-8921003182a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:20.664134 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-abb20915-c474-4b53-bbf6-8921003182a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:20.686543 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:20.758476 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb20915-c474-4b53-bbf6-8921003182a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:20.760085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb20915-c474-4b53-bbf6-8921003182a6 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:20.779790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 673/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:21:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:21:23.168229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecc4f5-82a0-4f69-a3ed-c4ab3891a336 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:23.169144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecc4f5-82a0-4f69-a3ed-c4ab3891a336 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:23.169428 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecc4f5-82a0-4f69-a3ed-c4ab3891a336 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:23.170995 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 691/1364] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:21:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:23.177614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571c9e0-91df-442d-a4c5-1bd84bde11a9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:23.178455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571c9e0-91df-442d-a4c5-1bd84bde11a9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:23.178900 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571c9e0-91df-442d-a4c5-1bd84bde11a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:23.251389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c571c9e0-91df-442d-a4c5-1bd84bde11a9 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:23.252788 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c571c9e0-91df-442d-a4c5-1bd84bde11a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:23.353263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:23.356888 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 674/1365] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:23.402171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718de22-c6a7-44ab-b139-7943170c6c45 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:23.402468 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718de22-c6a7-44ab-b139-7943170c6c45 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:23.402768 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718de22-c6a7-44ab-b139-7943170c6c45 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:23.404374 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 692/1366] 10.222.0.33 () {54 vars in 962 bytes} [Mon Jul 18 10:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:23.410942 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70057f5-d0b6-43ba-80cb-b5bcac15eb00 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:23.411291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70057f5-d0b6-43ba-80cb-b5bcac15eb00 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:23.411664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70057f5-d0b6-43ba-80cb-b5bcac15eb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:23.505770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c70057f5-d0b6-43ba-80cb-b5bcac15eb00 None None] MFA Rules not processed for user `29048c2021e340b081ae44a390583e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:23.507174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c70057f5-d0b6-43ba-80cb-b5bcac15eb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:23.641565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 675/1367] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Jul 18 10:21:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:23.649480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718de22-c6a7-44ab-b139-7943170c6c45 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:23.651193 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2718de22-c6a7-44ab-b139-7943170c6c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:23.806210 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718de22-c6a7-44ab-b139-7943170c6c45 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:23.828341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:23.926858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c2478-d6b0-42e2-b68f-561862f03583 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:23.927510 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c2478-d6b0-42e2-b68f-561862f03583 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:23.927794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c2478-d6b0-42e2-b68f-561862f03583 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:23.928071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c2478-d6b0-42e2-b68f-561862f03583 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:23.929766 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-895c2478-d6b0-42e2-b68f-561862f03583 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:24.027123 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c2478-d6b0-42e2-b68f-561862f03583 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:24.028772 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c2478-d6b0-42e2-b68f-561862f03583 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:24.047572 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 693/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:21:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:21:25.027326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5007d0-c56d-4686-8fe3-2a1802e7e7c9 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:25.028031 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5007d0-c56d-4686-8fe3-2a1802e7e7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:25.028588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5007d0-c56d-4686-8fe3-2a1802e7e7c9 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:25.030404 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 676/1369] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:21:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:25.035945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08358a4-393c-4e2a-995b-3493830e8f65 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:25.036396 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08358a4-393c-4e2a-995b-3493830e8f65 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:25.036524 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08358a4-393c-4e2a-995b-3493830e8f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:25.069339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:25.108358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c08358a4-393c-4e2a-995b-3493830e8f65 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:25.110268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c08358a4-393c-4e2a-995b-3493830e8f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:25.223256 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 694/1370] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:25.565574 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606249c0-8fbe-4ebe-9f20-e990b96e4181 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:25.565574 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606249c0-8fbe-4ebe-9f20-e990b96e4181 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:25.565574 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606249c0-8fbe-4ebe-9f20-e990b96e4181 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:25.566706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 677/1371] 10.222.0.33 () {54 vars in 964 bytes} [Mon Jul 18 10:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:25.571465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a70e0-5787-4526-9b73-4d7050443870 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:25.571757 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a70e0-5787-4526-9b73-4d7050443870 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:25.572036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a70e0-5787-4526-9b73-4d7050443870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:25.651113 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-583a70e0-5787-4526-9b73-4d7050443870 None None] MFA Rules not processed for user `20e29c37dc924514a882f936f5936ae1`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:25.652315 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-583a70e0-5787-4526-9b73-4d7050443870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:25.700664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:25.731546 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 695/1372] 10.222.0.33 () {58 vars in 1061 bytes} [Mon Jul 18 10:21:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:25.739601 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606249c0-8fbe-4ebe-9f20-e990b96e4181 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:25.741130 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-606249c0-8fbe-4ebe-9f20-e990b96e4181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:25.798945 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:25.857421 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606249c0-8fbe-4ebe-9f20-e990b96e4181 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:25.934468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf28d5e-67e0-4d0d-950c-e61be8d7f6fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:25.935112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf28d5e-67e0-4d0d-950c-e61be8d7f6fa service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:25.935400 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf28d5e-67e0-4d0d-950c-e61be8d7f6fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:25.935666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf28d5e-67e0-4d0d-950c-e61be8d7f6fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:25.937326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-acf28d5e-67e0-4d0d-950c-e61be8d7f6fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:26.047037 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:26.061771 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf28d5e-67e0-4d0d-950c-e61be8d7f6fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:26.063894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf28d5e-67e0-4d0d-950c-e61be8d7f6fa service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:26.084454 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 678/1373] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:21:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:21:26.145216 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597ef36-c028-4d0f-8f19-0d4064487696 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:26.145679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597ef36-c028-4d0f-8f19-0d4064487696 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:26.145774 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597ef36-c028-4d0f-8f19-0d4064487696 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:26.147311 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 696/1374] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:26.152368 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40038320-7b39-426c-b6da-2288b9f820e9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:26.152621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40038320-7b39-426c-b6da-2288b9f820e9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:26.152902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40038320-7b39-426c-b6da-2288b9f820e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:26.226897 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-40038320-7b39-426c-b6da-2288b9f820e9 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:26.228098 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-40038320-7b39-426c-b6da-2288b9f820e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:26.328546 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 679/1375] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:26.372086 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597ef36-c028-4d0f-8f19-0d4064487696 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:26.373698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6597ef36-c028-4d0f-8f19-0d4064487696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:26.523216 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597ef36-c028-4d0f-8f19-0d4064487696 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:26.574190 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:26.636112 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc32920-28ba-44d9-8750-c04d8b689829 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:26.636783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc32920-28ba-44d9-8750-c04d8b689829 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:26.637455 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc32920-28ba-44d9-8750-c04d8b689829 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:26.637455 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc32920-28ba-44d9-8750-c04d8b689829 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:26.638955 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4cc32920-28ba-44d9-8750-c04d8b689829 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:26.728581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc32920-28ba-44d9-8750-c04d8b689829 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:26.730194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc32920-28ba-44d9-8750-c04d8b689829 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:26.749723 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 697/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:21:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:21:28.795440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421aa9a2-754c-4054-a635-274e971735a5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:28.795958 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421aa9a2-754c-4054-a635-274e971735a5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:28.795958 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421aa9a2-754c-4054-a635-274e971735a5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:28.797373 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 680/1377] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:28.802456 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6f668-1d82-4baf-b60d-2231ff927724 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:28.802905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6f668-1d82-4baf-b60d-2231ff927724 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:28.803073 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6f668-1d82-4baf-b60d-2231ff927724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:28.872870 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-83b6f668-1d82-4baf-b60d-2231ff927724 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:28.875632 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-83b6f668-1d82-4baf-b60d-2231ff927724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:28.883111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:28.988707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 698/1378] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:29.085372 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421aa9a2-754c-4054-a635-274e971735a5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:29.087993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-421aa9a2-754c-4054-a635-274e971735a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:29.154645 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:29.242839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421aa9a2-754c-4054-a635-274e971735a5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:29.361110 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aeea2c-fe6c-447b-b23c-77679c9d3eb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:29.361760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aeea2c-fe6c-447b-b23c-77679c9d3eb8 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:29.362045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aeea2c-fe6c-447b-b23c-77679c9d3eb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:29.362324 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aeea2c-fe6c-447b-b23c-77679c9d3eb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:29.363957 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-78aeea2c-fe6c-447b-b23c-77679c9d3eb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:29.382689 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:29.468411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aeea2c-fe6c-447b-b23c-77679c9d3eb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:29.470619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aeea2c-fe6c-447b-b23c-77679c9d3eb8 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:29.494017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 681/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:21:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:21:31.568483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92adf0-92b9-4930-bfa7-fa5269db1475 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:31.569040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92adf0-92b9-4930-bfa7-fa5269db1475 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:31.569040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92adf0-92b9-4930-bfa7-fa5269db1475 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:31.570528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 699/1380] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:31.575926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a5dd5-2782-4b2e-baa2-1d7626369097 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:31.576471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a5dd5-2782-4b2e-baa2-1d7626369097 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:31.576471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a5dd5-2782-4b2e-baa2-1d7626369097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:31.651434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1c8a5dd5-2782-4b2e-baa2-1d7626369097 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:31.652666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1c8a5dd5-2782-4b2e-baa2-1d7626369097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:31.749247 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:31.756997 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 682/1381] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:31.800805 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92adf0-92b9-4930-bfa7-fa5269db1475 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:31.802619 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7e92adf0-92b9-4930-bfa7-fa5269db1475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:31.964094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:31.976281 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92adf0-92b9-4930-bfa7-fa5269db1475 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:32.104005 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94710bd5-488b-4b5e-8ccc-d7c85f4d40d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:32.104606 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94710bd5-488b-4b5e-8ccc-d7c85f4d40d3 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:32.104947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94710bd5-488b-4b5e-8ccc-d7c85f4d40d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:32.105231 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94710bd5-488b-4b5e-8ccc-d7c85f4d40d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:32.107196 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-94710bd5-488b-4b5e-8ccc-d7c85f4d40d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:32.210279 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94710bd5-488b-4b5e-8ccc-d7c85f4d40d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:32.212361 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94710bd5-488b-4b5e-8ccc-d7c85f4d40d3 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:32.224325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:32.235502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 700/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:21:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:21:34.637702 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc016a9-b114-4bbe-80ca-624bae3c625a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:34.638266 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc016a9-b114-4bbe-80ca-624bae3c625a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:34.638388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc016a9-b114-4bbe-80ca-624bae3c625a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:34.639780 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 683/1383] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:34.646654 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e529e-e1f3-4887-91c1-98df1386d8ea None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:34.647074 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e529e-e1f3-4887-91c1-98df1386d8ea None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:34.647310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e529e-e1f3-4887-91c1-98df1386d8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:34.714859 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-be1e529e-e1f3-4887-91c1-98df1386d8ea None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:34.716085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-be1e529e-e1f3-4887-91c1-98df1386d8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:34.818521 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 701/1384] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:21:34.864067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc016a9-b114-4bbe-80ca-624bae3c625a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:34.865674 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5cc016a9-b114-4bbe-80ca-624bae3c625a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:35.024808 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc016a9-b114-4bbe-80ca-624bae3c625a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:35.096936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:35.154741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9f530-db43-4cd8-a940-1dd26fb88196 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:35.155401 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9f530-db43-4cd8-a940-1dd26fb88196 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:35.156695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9f530-db43-4cd8-a940-1dd26fb88196 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:35.156695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9f530-db43-4cd8-a940-1dd26fb88196 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:35.158174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b2e9f530-db43-4cd8-a940-1dd26fb88196 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:35.251485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9f530-db43-4cd8-a940-1dd26fb88196 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:35.253165 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9f530-db43-4cd8-a940-1dd26fb88196 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:35.273315 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 684/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:21:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:21:37.664959 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0579ebe-f426-46ae-a3d3-68a33cab5cd7 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:37.665684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0579ebe-f426-46ae-a3d3-68a33cab5cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:37.665684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0579ebe-f426-46ae-a3d3-68a33cab5cd7 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:37.667000 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 702/1386] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:37.673876 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94216e40-5e3a-455f-a7cb-3bfcce992a04 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:37.674366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94216e40-5e3a-455f-a7cb-3bfcce992a04 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:37.674566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94216e40-5e3a-455f-a7cb-3bfcce992a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:37.748370 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-94216e40-5e3a-455f-a7cb-3bfcce992a04 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:37.750591 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-94216e40-5e3a-455f-a7cb-3bfcce992a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:37.790721 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:37.865469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 685/1387] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:37.918073 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0579ebe-f426-46ae-a3d3-68a33cab5cd7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:37.920005 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c0579ebe-f426-46ae-a3d3-68a33cab5cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:38.003579 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:38.082191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0579ebe-f426-46ae-a3d3-68a33cab5cd7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:38.213024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70efc9c7-d8c4-488a-a036-f9fff58790f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:38.213812 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efc9c7-d8c4-488a-a036-f9fff58790f5 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:38.214209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efc9c7-d8c4-488a-a036-f9fff58790f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:38.214509 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efc9c7-d8c4-488a-a036-f9fff58790f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:38.216713 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-70efc9c7-d8c4-488a-a036-f9fff58790f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:38.261065 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:38.317197 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70efc9c7-d8c4-488a-a036-f9fff58790f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:38.318811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70efc9c7-d8c4-488a-a036-f9fff58790f5 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:38.337857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 703/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:21:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:21:40.588283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f59cd-1ae5-4a4b-9f30-4b7c7612f00d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:40.588825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f59cd-1ae5-4a4b-9f30-4b7c7612f00d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:40.588825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f59cd-1ae5-4a4b-9f30-4b7c7612f00d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:40.590171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 686/1389] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:40.595978 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5139085-7b6a-4bc5-b873-8852e7e21fa1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:40.596428 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5139085-7b6a-4bc5-b873-8852e7e21fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:40.596561 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5139085-7b6a-4bc5-b873-8852e7e21fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:40.666518 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f5139085-7b6a-4bc5-b873-8852e7e21fa1 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:40.667729 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f5139085-7b6a-4bc5-b873-8852e7e21fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:40.763434 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 704/1390] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:40.808176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f59cd-1ae5-4a4b-9f30-4b7c7612f00d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:40.809714 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ea8f59cd-1ae5-4a4b-9f30-4b7c7612f00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:41.072524 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f59cd-1ae5-4a4b-9f30-4b7c7612f00d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:41.089870 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:41.191227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33fdf52-e8ce-410c-afcf-bf57ed8444f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:41.191880 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fdf52-e8ce-410c-afcf-bf57ed8444f1 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:41.192161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fdf52-e8ce-410c-afcf-bf57ed8444f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:41.192429 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fdf52-e8ce-410c-afcf-bf57ed8444f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:41.194105 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e33fdf52-e8ce-410c-afcf-bf57ed8444f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:41.281632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33fdf52-e8ce-410c-afcf-bf57ed8444f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:41.283219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33fdf52-e8ce-410c-afcf-bf57ed8444f1 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:41.301149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 687/1391] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:21:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:21:41.569610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e2786-3f5f-4bcc-bc22-254fbca77e13 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:41.569895 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e2786-3f5f-4bcc-bc22-254fbca77e13 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:41.570174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e2786-3f5f-4bcc-bc22-254fbca77e13 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:41.571592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 705/1392] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:41.576926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ff78b-ee2d-4c7b-92a8-05af9e2ff9e1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:41.577169 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ff78b-ee2d-4c7b-92a8-05af9e2ff9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:41.577457 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ff78b-ee2d-4c7b-92a8-05af9e2ff9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:41.603112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:41.645680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-587ff78b-ee2d-4c7b-92a8-05af9e2ff9e1 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:41.646923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-587ff78b-ee2d-4c7b-92a8-05af9e2ff9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:41.742866 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 688/1393] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:41.797852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543bef3b-692e-425f-b98b-53e8e45feac2 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:41.798330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543bef3b-692e-425f-b98b-53e8e45feac2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:41.798448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543bef3b-692e-425f-b98b-53e8e45feac2 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:41.799797 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 706/1394] 10.222.0.33 () {54 vars in 961 bytes} [Mon Jul 18 10:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:41.805063 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7eea9-f2e4-4005-a86c-b1f480be73f1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:41.805356 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7eea9-f2e4-4005-a86c-b1f480be73f1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:41.805636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7eea9-f2e4-4005-a86c-b1f480be73f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:41.879449 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bfb7eea9-f2e4-4005-a86c-b1f480be73f1 None None] MFA Rules not processed for user `3cee4d23797e430e9eeddc394ed7f50e`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:41.880672 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bfb7eea9-f2e4-4005-a86c-b1f480be73f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:41.921095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:41.957784 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 689/1395] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Jul 18 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:41.965390 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543bef3b-692e-425f-b98b-53e8e45feac2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:41.966899 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-543bef3b-692e-425f-b98b-53e8e45feac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:41.981547 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:42.076772 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543bef3b-692e-425f-b98b-53e8e45feac2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:42.149085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabdffc-1962-47af-acbb-1df90f31d536 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cee4d23797e430e9eeddc394ed7f50e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:42.149650 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabdffc-1962-47af-acbb-1df90f31d536 service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:42.149936 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabdffc-1962-47af-acbb-1df90f31d536 service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:42.150214 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabdffc-1962-47af-acbb-1df90f31d536 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:42.151863 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8fabdffc-1962-47af-acbb-1df90f31d536 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:42.231657 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:42.271736 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabdffc-1962-47af-acbb-1df90f31d536 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:42.273213 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabdffc-1962-47af-acbb-1df90f31d536 service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:42.291456 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 707/1396] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:21:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:21:42.994224 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce925fe2-a135-4e8e-95a7-21f601991778 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:42.994875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce925fe2-a135-4e8e-95a7-21f601991778 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:42.994875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce925fe2-a135-4e8e-95a7-21f601991778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:43.059642 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ce925fe2-a135-4e8e-95a7-21f601991778 None None] MFA Rules not processed for user `a09029144c454b099bea86aaa5229ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:43.060822 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ce925fe2-a135-4e8e-95a7-21f601991778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:43.166321 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 690/1397] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 18 10:21:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:43.210497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c604333-11d9-48da-b664-6073a200e028 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:43.210997 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c604333-11d9-48da-b664-6073a200e028 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:43.211097 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c604333-11d9-48da-b664-6073a200e028 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:43.212628 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 708/1398] 10.222.0.33 () {54 vars in 943 bytes} [Mon Jul 18 10:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:43.218959 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585df40-a3ec-42ea-a74d-ca1104937109 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:43.219315 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585df40-a3ec-42ea-a74d-ca1104937109 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:43.219656 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585df40-a3ec-42ea-a74d-ca1104937109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:43.287594 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:43.297290 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d585df40-a3ec-42ea-a74d-ca1104937109 None None] MFA Rules not processed for user `fd6168873f2846c8af73b903bba17eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:43.298512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d585df40-a3ec-42ea-a74d-ca1104937109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:43.388476 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d39bd-7126-4af2-b73e-1f34a46667e3 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:43.388798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d39bd-7126-4af2-b73e-1f34a46667e3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:43.389089 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d39bd-7126-4af2-b73e-1f34a46667e3 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:43.391048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 709/1399] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:21:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:43.395912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96c58f-f216-478e-acf8-3d1b241751b4 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:43.396204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96c58f-f216-478e-acf8-3d1b241751b4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:43.396481 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96c58f-f216-478e-acf8-3d1b241751b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:43.407035 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 691/1400] 10.222.0.33 () {58 vars in 1040 bytes} [Mon Jul 18 10:21:43 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:21:43.417418 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585df40-a3ec-42ea-a74d-ca1104937109 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:43.426904 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585df40-a3ec-42ea-a74d-ca1104937109 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:43.462222 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7a96c58f-f216-478e-acf8-3d1b241751b4 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:43.463439 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7a96c58f-f216-478e-acf8-3d1b241751b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:43.531758 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c7e70-876a-40a0-869f-df68a86b86bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6168873f2846c8af73b903bba17eb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:43.532410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c7e70-876a-40a0-869f-df68a86b86bd service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:43.532729 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c7e70-876a-40a0-869f-df68a86b86bd service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:43.533012 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c7e70-876a-40a0-869f-df68a86b86bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:43.534813 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1c4c7e70-876a-40a0-869f-df68a86b86bd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:43.537356 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:43.562992 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 710/1401] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:21:43.608246 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96c58f-f216-478e-acf8-3d1b241751b4 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:43.610026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7a96c58f-f216-478e-acf8-3d1b241751b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:43.630974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c7e70-876a-40a0-869f-df68a86b86bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:43.632600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c7e70-876a-40a0-869f-df68a86b86bd service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:43.636671 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:43.641416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 692/1402] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 18 10:21:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 10:21:43.729916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96c58f-f216-478e-acf8-3d1b241751b4 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:43.871189 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855f2ae-61a9-4082-a4de-c417f4f94388 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:43.871849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855f2ae-61a9-4082-a4de-c417f4f94388 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:43.872137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855f2ae-61a9-4082-a4de-c417f4f94388 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:43.872416 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855f2ae-61a9-4082-a4de-c417f4f94388 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:43.874249 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2855f2ae-61a9-4082-a4de-c417f4f94388 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:43.896033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:43.976232 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855f2ae-61a9-4082-a4de-c417f4f94388 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:43.977895 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855f2ae-61a9-4082-a4de-c417f4f94388 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:44.017128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 711/1403] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:21:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:21:46.118540 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb4c26-1a7f-4481-a800-9e762dcc2428 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:46.119142 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb4c26-1a7f-4481-a800-9e762dcc2428 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:46.119142 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb4c26-1a7f-4481-a800-9e762dcc2428 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:46.120494 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 693/1404] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:46.126507 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376491a-963d-444c-ad74-a64928d57e4d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:46.127404 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376491a-963d-444c-ad74-a64928d57e4d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:46.127759 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376491a-963d-444c-ad74-a64928d57e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:46.226517 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a376491a-963d-444c-ad74-a64928d57e4d None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:46.228249 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a376491a-963d-444c-ad74-a64928d57e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:46.355482 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:46.368707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 712/1405] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:21:46.423737 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb4c26-1a7f-4481-a800-9e762dcc2428 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:46.425405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bddb4c26-1a7f-4481-a800-9e762dcc2428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:46.612238 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:46.615568 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb4c26-1a7f-4481-a800-9e762dcc2428 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:46.776891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d891d-a386-4b26-9710-7040b4a2cabf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:46.777703 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d891d-a386-4b26-9710-7040b4a2cabf service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:46.778227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d891d-a386-4b26-9710-7040b4a2cabf service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:46.778632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d891d-a386-4b26-9710-7040b4a2cabf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:46.780463 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bd7d891d-a386-4b26-9710-7040b4a2cabf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:46.897923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d891d-a386-4b26-9710-7040b4a2cabf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:46.899833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d891d-a386-4b26-9710-7040b4a2cabf service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:46.920850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 694/1406] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:21:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:21:48.967745 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353a612-1afa-4d6d-93e0-343dc383f455 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:48.968308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353a612-1afa-4d6d-93e0-343dc383f455 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:48.968308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353a612-1afa-4d6d-93e0-343dc383f455 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:48.975157 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b84b04-76e1-4c14-8ffd-5cdf2f876d31 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:48.975666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b84b04-76e1-4c14-8ffd-5cdf2f876d31 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:48.975769 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b84b04-76e1-4c14-8ffd-5cdf2f876d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:48.979221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 713/1407] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:21:48 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:21:48.979370 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:49.052876 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-49b84b04-76e1-4c14-8ffd-5cdf2f876d31 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:21:49.052876 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-49b84b04-76e1-4c14-8ffd-5cdf2f876d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:49.163602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 695/1408] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:21:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:21:49.208769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353a612-1afa-4d6d-93e0-343dc383f455 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:21:49.210385 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4353a612-1afa-4d6d-93e0-343dc383f455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:49.389215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353a612-1afa-4d6d-93e0-343dc383f455 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:21:49.463420 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:49.517561 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c833d-43ec-4aac-aed1-2c5cd18fb6a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:21:49.518398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c833d-43ec-4aac-aed1-2c5cd18fb6a7 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:21:49.518914 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c833d-43ec-4aac-aed1-2c5cd18fb6a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:21:49.519296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c833d-43ec-4aac-aed1-2c5cd18fb6a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:21:49.521206 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f41c833d-43ec-4aac-aed1-2c5cd18fb6a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:21:49.627169 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c833d-43ec-4aac-aed1-2c5cd18fb6a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:21:49.629123 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c833d-43ec-4aac-aed1-2c5cd18fb6a7 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:21:49.651954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 714/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:21:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:21:59.159132 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=64 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:21:59.649169 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=64 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:05.753506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29712f36-3ecf-456a-8a26-feb7396e7e67 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:05.754086 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29712f36-3ecf-456a-8a26-feb7396e7e67 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:05.754086 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29712f36-3ecf-456a-8a26-feb7396e7e67 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:05.755471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 696/1410] 10.222.0.33 () {54 vars in 918 bytes} [Mon Jul 18 10:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:05.760805 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bbae8-3a58-484b-a76c-7a599ef35796 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:05.761287 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bbae8-3a58-484b-a76c-7a599ef35796 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:05.761404 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bbae8-3a58-484b-a76c-7a599ef35796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:05.834568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f30bbae8-3a58-484b-a76c-7a599ef35796 None None] MFA Rules not processed for user `d07165f554a445169b2c3f0d173c0bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:05.835774 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f30bbae8-3a58-484b-a76c-7a599ef35796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:05.941255 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 715/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Jul 18 10:22:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:05.955805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93d404-1692-43f2-9ad1-a6854fb6430b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:05.956103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93d404-1692-43f2-9ad1-a6854fb6430b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:05.956386 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93d404-1692-43f2-9ad1-a6854fb6430b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:05.957847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 697/1412] 10.222.0.33 () {54 vars in 918 bytes} [Mon Jul 18 10:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:05.962481 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a4f9d-b054-4605-b947-50050c2f6422 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:05.962771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a4f9d-b054-4605-b947-50050c2f6422 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:05.963048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a4f9d-b054-4605-b947-50050c2f6422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:06.033204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:06.035788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-519a4f9d-b054-4605-b947-50050c2f6422 None None] MFA Rules not processed for user `29048c2021e340b081ae44a390583e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:06.037612 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-519a4f9d-b054-4605-b947-50050c2f6422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:06.153863 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 716/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Jul 18 10:22:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:08.599595 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b3376-a049-4c82-a390-98ca97a5a204 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:08.600131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b3376-a049-4c82-a390-98ca97a5a204 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:08.600131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b3376-a049-4c82-a390-98ca97a5a204 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:08.601555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:08.607070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a858f8a-a8f2-4cf9-ad43-9a8c5dcdf2c6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:08.607474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a858f8a-a8f2-4cf9-ad43-9a8c5dcdf2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:08.607637 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a858f8a-a8f2-4cf9-ad43-9a8c5dcdf2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:08.671206 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0a858f8a-a8f2-4cf9-ad43-9a8c5dcdf2c6 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:08.672414 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0a858f8a-a8f2-4cf9-ad43-9a8c5dcdf2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:08.726206 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:08.767884 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:08.812439 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b3376-a049-4c82-a390-98ca97a5a204 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:08.814085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f14b3376-a049-4c82-a390-98ca97a5a204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:08.955974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b3376-a049-4c82-a390-98ca97a5a204 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:09.031019 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:09.071740 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d6202-4b45-4371-998c-654cb69bc08a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:09.072391 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d6202-4b45-4371-998c-654cb69bc08a service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:09.072708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d6202-4b45-4371-998c-654cb69bc08a service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:09.072983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d6202-4b45-4371-998c-654cb69bc08a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:09.074635 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ca0d6202-4b45-4371-998c-654cb69bc08a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:09.166590 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d6202-4b45-4371-998c-654cb69bc08a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:09.168181 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d6202-4b45-4371-998c-654cb69bc08a service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:09.187300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 699/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:10.095395 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc935d-8679-42f5-b214-87eda30a4373 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:10.095972 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc935d-8679-42f5-b214-87eda30a4373 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:10.095972 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc935d-8679-42f5-b214-87eda30a4373 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:10.097407 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:10.102558 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb223d-831a-4b21-b622-bab8bcbb5514 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:10.102947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb223d-831a-4b21-b622-bab8bcbb5514 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:10.103131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb223d-831a-4b21-b622-bab8bcbb5514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:10.178057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e9eb223d-831a-4b21-b622-bab8bcbb5514 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:10.179258 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e9eb223d-831a-4b21-b622-bab8bcbb5514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:10.206432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:10.276267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:10.428706 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc935d-8679-42f5-b214-87eda30a4373 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:10.430237 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-eefc935d-8679-42f5-b214-87eda30a4373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:10.579463 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc935d-8679-42f5-b214-87eda30a4373 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:10.618706 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:10.692304 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75636e4a-47a7-46ed-bbe8-c3ca5647b72e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:10.692930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75636e4a-47a7-46ed-bbe8-c3ca5647b72e service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:10.693221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75636e4a-47a7-46ed-bbe8-c3ca5647b72e service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:10.693502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75636e4a-47a7-46ed-bbe8-c3ca5647b72e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:10.695186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-75636e4a-47a7-46ed-bbe8-c3ca5647b72e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:10.783308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75636e4a-47a7-46ed-bbe8-c3ca5647b72e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:10.784947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75636e4a-47a7-46ed-bbe8-c3ca5647b72e service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:10.803187 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 719/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:11.722379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e44ebc-2bf1-4439-bdd2-b67618c76e9c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:11.722921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e44ebc-2bf1-4439-bdd2-b67618c76e9c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:11.723022 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e44ebc-2bf1-4439-bdd2-b67618c76e9c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:11.724318 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:11.729522 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5365add-3503-43ec-b8ff-3326ca7c2223 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:11.729965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5365add-3503-43ec-b8ff-3326ca7c2223 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:11.730071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5365add-3503-43ec-b8ff-3326ca7c2223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:11.783745 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:11.804971 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c5365add-3503-43ec-b8ff-3326ca7c2223 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:11.806193 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c5365add-3503-43ec-b8ff-3326ca7c2223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:11.900944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:12.052382 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e44ebc-2bf1-4439-bdd2-b67618c76e9c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:12.054550 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-26e44ebc-2bf1-4439-bdd2-b67618c76e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:12.197517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e44ebc-2bf1-4439-bdd2-b67618c76e9c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:12.209253 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:12.308392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa988d36-d037-4094-b685-4966773053e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:12.308994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa988d36-d037-4094-b685-4966773053e9 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:12.309282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa988d36-d037-4094-b685-4966773053e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:12.309588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa988d36-d037-4094-b685-4966773053e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:12.311750 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fa988d36-d037-4094-b685-4966773053e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:12.399973 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa988d36-d037-4094-b685-4966773053e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:12.401677 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa988d36-d037-4094-b685-4966773053e9 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:12.419754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 702/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:13.277843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ec79e-d99b-4e70-a41c-fb2e073b6759 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:13.278400 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ec79e-d99b-4e70-a41c-fb2e073b6759 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:13.278400 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ec79e-d99b-4e70-a41c-fb2e073b6759 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:13.279795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:13.284670 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b217f4f-04a2-479f-9a43-ea531f467072 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:13.285057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b217f4f-04a2-479f-9a43-ea531f467072 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:13.285272 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b217f4f-04a2-479f-9a43-ea531f467072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:13.303245 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:13.359142 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5b217f4f-04a2-479f-9a43-ea531f467072 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:13.360271 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5b217f4f-04a2-479f-9a43-ea531f467072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:13.453594 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:22:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:13.603308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ec79e-d99b-4e70-a41c-fb2e073b6759 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:13.604843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f80ec79e-d99b-4e70-a41c-fb2e073b6759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:13.718622 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:13.755904 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ec79e-d99b-4e70-a41c-fb2e073b6759 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:13.869452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4ed0e-93aa-4910-888f-6b8176f0373a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:13.870095 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4ed0e-93aa-4910-888f-6b8176f0373a service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:13.870378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4ed0e-93aa-4910-888f-6b8176f0373a service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:13.870658 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4ed0e-93aa-4910-888f-6b8176f0373a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:13.872293 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8dd4ed0e-93aa-4910-888f-6b8176f0373a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:13.946153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:13.960224 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd4ed0e-93aa-4910-888f-6b8176f0373a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:13.961886 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd4ed0e-93aa-4910-888f-6b8176f0373a service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:13.979435 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 722/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:14.836519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb28b5-c914-4649-aa05-16b83f86cbae None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:14.837081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb28b5-c914-4649-aa05-16b83f86cbae None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:14.837081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb28b5-c914-4649-aa05-16b83f86cbae None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:14.838483 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:14.843322 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca22dd7-d35c-4f88-b88b-419615ece6ec None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:14.843771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca22dd7-d35c-4f88-b88b-419615ece6ec None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:14.843939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca22dd7-d35c-4f88-b88b-419615ece6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:14.921769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-dca22dd7-d35c-4f88-b88b-419615ece6ec None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:14.922894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dca22dd7-d35c-4f88-b88b-419615ece6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:15.017326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:15.165322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb28b5-c914-4649-aa05-16b83f86cbae None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:15.166837 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e6cb28b5-c914-4649-aa05-16b83f86cbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:15.251118 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:15.311961 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb28b5-c914-4649-aa05-16b83f86cbae None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:15.421061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84432571-df98-443e-9bf5-1dd95e3af200 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:15.421697 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84432571-df98-443e-9bf5-1dd95e3af200 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:15.421977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84432571-df98-443e-9bf5-1dd95e3af200 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:15.422249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84432571-df98-443e-9bf5-1dd95e3af200 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:15.423869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-84432571-df98-443e-9bf5-1dd95e3af200 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:15.470366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:15.512440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84432571-df98-443e-9bf5-1dd95e3af200 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:15.513977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84432571-df98-443e-9bf5-1dd95e3af200 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:15.531832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 705/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:16.341513 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cb0e6-0062-4d0e-a5f0-b47b06314008 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:16.342076 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cb0e6-0062-4d0e-a5f0-b47b06314008 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:16.342076 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cb0e6-0062-4d0e-a5f0-b47b06314008 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:16.343453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:16.348252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d879e90-d255-4d5a-8072-d68f2bf08f45 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:16.348710 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d879e90-d255-4d5a-8072-d68f2bf08f45 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:16.348852 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d879e90-d255-4d5a-8072-d68f2bf08f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:16.421864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3d879e90-d255-4d5a-8072-d68f2bf08f45 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:16.422992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3d879e90-d255-4d5a-8072-d68f2bf08f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:16.519147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:22:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:16.670161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cb0e6-0062-4d0e-a5f0-b47b06314008 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:16.672235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e46cb0e6-0062-4d0e-a5f0-b47b06314008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:16.719844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:16.815968 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cb0e6-0062-4d0e-a5f0-b47b06314008 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:16.927824 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be709021-0ded-427c-bfbf-80fb9e25599e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:16.928402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be709021-0ded-427c-bfbf-80fb9e25599e service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:16.928720 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be709021-0ded-427c-bfbf-80fb9e25599e service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:16.928996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be709021-0ded-427c-bfbf-80fb9e25599e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:16.930637 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-be709021-0ded-427c-bfbf-80fb9e25599e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:16.937552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:17.020074 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be709021-0ded-427c-bfbf-80fb9e25599e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:17.021697 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be709021-0ded-427c-bfbf-80fb9e25599e service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:17.039887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 725/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:17.847607 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1dc793-f68c-435f-a7ae-32a8a5d4bb3b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:17.848170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1dc793-f68c-435f-a7ae-32a8a5d4bb3b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:17.848170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1dc793-f68c-435f-a7ae-32a8a5d4bb3b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:17.849608 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:17.854493 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7fa22-d478-45d2-9ebf-e65a3e67b4f1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:17.854974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7fa22-d478-45d2-9ebf-e65a3e67b4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:17.855076 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7fa22-d478-45d2-9ebf-e65a3e67b4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:17.928413 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ded7fa22-d478-45d2-9ebf-e65a3e67b4f1 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:17.929636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ded7fa22-d478-45d2-9ebf-e65a3e67b4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:17.999134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:18.027323 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:18.179093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1dc793-f68c-435f-a7ae-32a8a5d4bb3b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:18.180625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ef1dc793-f68c-435f-a7ae-32a8a5d4bb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:18.198321 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:18.321959 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1dc793-f68c-435f-a7ae-32a8a5d4bb3b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:18.407033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:18.432232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bf74f-3d36-4905-88ff-79b08a82c613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:18.432901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bf74f-3d36-4905-88ff-79b08a82c613 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:18.433184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bf74f-3d36-4905-88ff-79b08a82c613 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:18.433454 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bf74f-3d36-4905-88ff-79b08a82c613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:18.435081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c58bf74f-3d36-4905-88ff-79b08a82c613 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:18.523117 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bf74f-3d36-4905-88ff-79b08a82c613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:18.524753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bf74f-3d36-4905-88ff-79b08a82c613 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:18.542817 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 708/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:19.348293 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eeaf42-5115-41b0-aa02-03ac18e3d9a7 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:19.348908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eeaf42-5115-41b0-aa02-03ac18e3d9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:19.348908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eeaf42-5115-41b0-aa02-03ac18e3d9a7 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:19.350289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:19.355322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a76963-10a0-4efb-aeeb-d675852ce183 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:19.355740 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a76963-10a0-4efb-aeeb-d675852ce183 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:19.356116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a76963-10a0-4efb-aeeb-d675852ce183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:19.427672 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e2a76963-10a0-4efb-aeeb-d675852ce183 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:19.428816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e2a76963-10a0-4efb-aeeb-d675852ce183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:19.469347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:19.525303 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:19.676693 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eeaf42-5115-41b0-aa02-03ac18e3d9a7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:19.678308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-69eeaf42-5115-41b0-aa02-03ac18e3d9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:19.822070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eeaf42-5115-41b0-aa02-03ac18e3d9a7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:19.876055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:19.934429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f6075-e7e8-4b12-90cc-6c85fd2a657f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:19.935001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f6075-e7e8-4b12-90cc-6c85fd2a657f service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:19.935285 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f6075-e7e8-4b12-90cc-6c85fd2a657f service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:19.935564 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f6075-e7e8-4b12-90cc-6c85fd2a657f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:19.937277 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-301f6075-e7e8-4b12-90cc-6c85fd2a657f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:20.028272 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f6075-e7e8-4b12-90cc-6c85fd2a657f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:20.030438 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f6075-e7e8-4b12-90cc-6c85fd2a657f service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:20.049082 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 728/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:20.850069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3fdc5-1dd1-4653-aaab-eb175867aeea None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:20.850690 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3fdc5-1dd1-4653-aaab-eb175867aeea None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:20.850690 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3fdc5-1dd1-4653-aaab-eb175867aeea None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:20.852033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:20.858345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb1376-c91f-49f4-ab6b-449ee68c6fe3 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:20.858832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb1376-c91f-49f4-ab6b-449ee68c6fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:20.858947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb1376-c91f-49f4-ab6b-449ee68c6fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:20.936209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2beb1376-c91f-49f4-ab6b-449ee68c6fe3 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:20.937452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2beb1376-c91f-49f4-ab6b-449ee68c6fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:20.939461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:21.033771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:21.191552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3fdc5-1dd1-4653-aaab-eb175867aeea None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:21.193009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-afc3fdc5-1dd1-4653-aaab-eb175867aeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:21.336665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3fdc5-1dd1-4653-aaab-eb175867aeea None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:21.363331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:21.448007 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3b636-5de1-4398-8c72-d33fd815f55f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:21.448642 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3b636-5de1-4398-8c72-d33fd815f55f service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:21.448936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3b636-5de1-4398-8c72-d33fd815f55f service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:21.449216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3b636-5de1-4398-8c72-d33fd815f55f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:21.450867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-32f3b636-5de1-4398-8c72-d33fd815f55f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:21.539701 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3b636-5de1-4398-8c72-d33fd815f55f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:21.541326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3b636-5de1-4398-8c72-d33fd815f55f service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:21.559071 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 711/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:23.472156 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4e9c7-ab2e-4cc8-a8b6-239c30ef0999 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:23.472755 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4e9c7-ab2e-4cc8-a8b6-239c30ef0999 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:23.472755 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4e9c7-ab2e-4cc8-a8b6-239c30ef0999 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:23.474159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:23.479508 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b652075-55f3-4c88-b3ca-e59298a98718 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:23.479918 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b652075-55f3-4c88-b3ca-e59298a98718 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:23.480055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b652075-55f3-4c88-b3ca-e59298a98718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:23.524464 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:23.549937 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0b652075-55f3-4c88-b3ca-e59298a98718 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:23.551158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0b652075-55f3-4c88-b3ca-e59298a98718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:23.645823 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:23.686139 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4e9c7-ab2e-4cc8-a8b6-239c30ef0999 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:23.687683 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-42c4e9c7-ab2e-4cc8-a8b6-239c30ef0999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:23.789665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4e9c7-ab2e-4cc8-a8b6-239c30ef0999 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:23.822143 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:23.863375 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab373d-4b99-4427-a72d-89f745dbdbb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:23.864023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab373d-4b99-4427-a72d-89f745dbdbb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:23.864306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab373d-4b99-4427-a72d-89f745dbdbb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:23.864612 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab373d-4b99-4427-a72d-89f745dbdbb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:23.866252 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-62ab373d-4b99-4427-a72d-89f745dbdbb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:23.972973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab373d-4b99-4427-a72d-89f745dbdbb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:23.974431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab373d-4b99-4427-a72d-89f745dbdbb8 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:23.994437 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 731/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:26.157678 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ce489-53b3-4011-8086-48107cc88962 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:26.158099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ce489-53b3-4011-8086-48107cc88962 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:26.158287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ce489-53b3-4011-8086-48107cc88962 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:26.159724 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:26.165108 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a52004-534b-4af8-ae98-cbb2d8d8627a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:26.165502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a52004-534b-4af8-ae98-cbb2d8d8627a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:26.165677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a52004-534b-4af8-ae98-cbb2d8d8627a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:26.229580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a2a52004-534b-4af8-ae98-cbb2d8d8627a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:26.231041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a2a52004-534b-4af8-ae98-cbb2d8d8627a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:26.243422 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:26.327123 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:26.365742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ce489-53b3-4011-8086-48107cc88962 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:26.367270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c4ce489-53b3-4011-8086-48107cc88962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:26.467864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ce489-53b3-4011-8086-48107cc88962 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:26.479621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:26.540023 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133b940-3727-4df0-8673-adf5ac7a728a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:26.540703 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133b940-3727-4df0-8673-adf5ac7a728a service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:26.540989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133b940-3727-4df0-8673-adf5ac7a728a service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:26.541267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133b940-3727-4df0-8673-adf5ac7a728a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:26.542897 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3133b940-3727-4df0-8673-adf5ac7a728a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:26.649517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3133b940-3727-4df0-8673-adf5ac7a728a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:26.650948 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3133b940-3727-4df0-8673-adf5ac7a728a service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:26.669688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 714/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:28.996174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914ff6c-5ff7-418f-80e5-81e4f95e3f4a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:28.996670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914ff6c-5ff7-418f-80e5-81e4f95e3f4a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:28.996769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914ff6c-5ff7-418f-80e5-81e4f95e3f4a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:28.998700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:22:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:22:29.003985 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00110e40-143c-43cb-9829-8734a87ff55b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:29.004415 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00110e40-143c-43cb-9829-8734a87ff55b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:29.004572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00110e40-143c-43cb-9829-8734a87ff55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:29.051430 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:29.070051 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-00110e40-143c-43cb-9829-8734a87ff55b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:29.071292 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-00110e40-143c-43cb-9829-8734a87ff55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:29.165805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:22:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:29.205726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914ff6c-5ff7-418f-80e5-81e4f95e3f4a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:29.207257 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d914ff6c-5ff7-418f-80e5-81e4f95e3f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:29.308889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914ff6c-5ff7-418f-80e5-81e4f95e3f4a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:29.348816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:29.382516 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d79eaf-1191-4f65-a0db-8b4707c48c2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:29.383160 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d79eaf-1191-4f65-a0db-8b4707c48c2a service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:29.383445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d79eaf-1191-4f65-a0db-8b4707c48c2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:29.383724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d79eaf-1191-4f65-a0db-8b4707c48c2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:29.385402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d9d79eaf-1191-4f65-a0db-8b4707c48c2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:29.493344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d79eaf-1191-4f65-a0db-8b4707c48c2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:29.494944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d79eaf-1191-4f65-a0db-8b4707c48c2a service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:29.512727 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 734/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:31.725314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84452855-ae17-4120-bc67-7d08d538f0d4 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:31.727295 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84452855-ae17-4120-bc67-7d08d538f0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:31.727295 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84452855-ae17-4120-bc67-7d08d538f0d4 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:31.727891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:22:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:31.733463 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814f269-017c-407c-8ada-bd7d7359b3ff None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:31.733914 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814f269-017c-407c-8ada-bd7d7359b3ff None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:31.734015 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814f269-017c-407c-8ada-bd7d7359b3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:31.798958 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1814f269-017c-407c-8ada-bd7d7359b3ff None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:31.800154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1814f269-017c-407c-8ada-bd7d7359b3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:31.819916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:31.895347 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:22:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:31.942202 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84452855-ae17-4120-bc67-7d08d538f0d4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:31.943750 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-84452855-ae17-4120-bc67-7d08d538f0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:32.044753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84452855-ae17-4120-bc67-7d08d538f0d4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:32.064582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:32.117879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1949d-084c-4d6b-aa2b-b3da4a440d73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:32.118526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1949d-084c-4d6b-aa2b-b3da4a440d73 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:32.118811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1949d-084c-4d6b-aa2b-b3da4a440d73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:32.119118 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1949d-084c-4d6b-aa2b-b3da4a440d73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:32.120792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-22a1949d-084c-4d6b-aa2b-b3da4a440d73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:32.228531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a1949d-084c-4d6b-aa2b-b3da4a440d73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:32.229967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a1949d-084c-4d6b-aa2b-b3da4a440d73 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:32.248140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 717/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:34.452333 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0b4f8-9db0-4371-a4d4-c4978471e808 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:34.452905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0b4f8-9db0-4371-a4d4-c4978471e808 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:34.453006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0b4f8-9db0-4371-a4d4-c4978471e808 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:34.454328 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:22:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:34.459705 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706a4fe-3171-4a50-9ace-64e5c65b4ced None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:34.460109 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706a4fe-3171-4a50-9ace-64e5c65b4ced None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:34.460258 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706a4fe-3171-4a50-9ace-64e5c65b4ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:34.522098 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:34.525107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8706a4fe-3171-4a50-9ace-64e5c65b4ced None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:34.526922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8706a4fe-3171-4a50-9ace-64e5c65b4ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:34.623811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:34.660836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0b4f8-9db0-4371-a4d4-c4978471e808 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:34.662559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8bd0b4f8-9db0-4371-a4d4-c4978471e808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:34.768842 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0b4f8-9db0-4371-a4d4-c4978471e808 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:34.823513 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:34.847921 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee75a7c-48c7-48d8-b6f7-afade24968ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:34.848609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee75a7c-48c7-48d8-b6f7-afade24968ae service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:34.848894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee75a7c-48c7-48d8-b6f7-afade24968ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:34.849170 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee75a7c-48c7-48d8-b6f7-afade24968ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:34.850797 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3ee75a7c-48c7-48d8-b6f7-afade24968ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:34.957121 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee75a7c-48c7-48d8-b6f7-afade24968ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:34.958552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee75a7c-48c7-48d8-b6f7-afade24968ae service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:34.976620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 737/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:37.039618 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6d409-49ac-4f60-9a40-cc166a514d76 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:37.040191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6d409-49ac-4f60-9a40-cc166a514d76 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:37.040191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6d409-49ac-4f60-9a40-cc166a514d76 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:37.041514 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:37.046895 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefb601-5b1f-4254-a26c-c9ef32fcd18b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:37.047317 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefb601-5b1f-4254-a26c-c9ef32fcd18b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:37.047467 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefb601-5b1f-4254-a26c-c9ef32fcd18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:37.112213 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-adefb601-5b1f-4254-a26c-c9ef32fcd18b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:37.113469 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-adefb601-5b1f-4254-a26c-c9ef32fcd18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:37.142990 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:37.212528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:22:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:37.250963 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6d409-49ac-4f60-9a40-cc166a514d76 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:37.252523 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a1e6d409-49ac-4f60-9a40-cc166a514d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:37.352597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6d409-49ac-4f60-9a40-cc166a514d76 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:37.380514 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:37.424362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa5691-315b-4a60-973a-d6177c9c30c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:37.425027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa5691-315b-4a60-973a-d6177c9c30c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:37.425310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa5691-315b-4a60-973a-d6177c9c30c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:37.425590 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa5691-315b-4a60-973a-d6177c9c30c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:37.427213 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f3fa5691-315b-4a60-973a-d6177c9c30c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:37.535718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa5691-315b-4a60-973a-d6177c9c30c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:37.537200 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa5691-315b-4a60-973a-d6177c9c30c9 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:37.555790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 720/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:39.618603 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bbc73-7452-47b2-bb31-75752e6b3bf1 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:39.619140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bbc73-7452-47b2-bb31-75752e6b3bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:39.619140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bbc73-7452-47b2-bb31-75752e6b3bf1 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:39.620560 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:39.626254 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c76369-e8bd-4311-9a35-03a81487ba96 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:39.626684 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c76369-e8bd-4311-9a35-03a81487ba96 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:39.626847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c76369-e8bd-4311-9a35-03a81487ba96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:39.690738 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f3c76369-e8bd-4311-9a35-03a81487ba96 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:39.691939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f3c76369-e8bd-4311-9a35-03a81487ba96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:39.700031 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:39.788096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:22:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:39.826796 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bbc73-7452-47b2-bb31-75752e6b3bf1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:39.828368 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-589bbc73-7452-47b2-bb31-75752e6b3bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:39.932150 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bbc73-7452-47b2-bb31-75752e6b3bf1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:39.988015 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:40.007433 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62913270-fa14-47bf-88fa-d3929a0530a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:40.008090 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62913270-fa14-47bf-88fa-d3929a0530a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:40.008376 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62913270-fa14-47bf-88fa-d3929a0530a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:40.008684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62913270-fa14-47bf-88fa-d3929a0530a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:40.010339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-62913270-fa14-47bf-88fa-d3929a0530a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:40.117080 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62913270-fa14-47bf-88fa-d3929a0530a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:40.118516 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62913270-fa14-47bf-88fa-d3929a0530a4 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:40.136133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 740/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:42.198842 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733b724-5624-4879-8665-547e1f2c9f10 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:42.199370 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733b724-5624-4879-8665-547e1f2c9f10 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:42.199468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733b724-5624-4879-8665-547e1f2c9f10 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:42.200856 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:42.206008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532c8be-d1f4-4a00-a93e-5145183e1f68 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:42.206431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532c8be-d1f4-4a00-a93e-5145183e1f68 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:42.206550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532c8be-d1f4-4a00-a93e-5145183e1f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:42.271132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6532c8be-d1f4-4a00-a93e-5145183e1f68 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:42.272338 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6532c8be-d1f4-4a00-a93e-5145183e1f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:42.309612 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:42.367586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:22:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:42.405474 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733b724-5624-4879-8665-547e1f2c9f10 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:42.407040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8733b724-5624-4879-8665-547e1f2c9f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:42.507732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733b724-5624-4879-8665-547e1f2c9f10 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:42.544308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:42.579503 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c389e-a3e1-4d4e-9932-7584e7b7f0fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:42.580139 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c389e-a3e1-4d4e-9932-7584e7b7f0fc service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:42.580422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c389e-a3e1-4d4e-9932-7584e7b7f0fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:42.580724 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c389e-a3e1-4d4e-9932-7584e7b7f0fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:42.582360 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-351c389e-a3e1-4d4e-9932-7584e7b7f0fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:42.688298 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c389e-a3e1-4d4e-9932-7584e7b7f0fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:42.689758 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c389e-a3e1-4d4e-9932-7584e7b7f0fc service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:42.708392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 723/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:44.714910 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41191928-bada-4441-9155-cf0f5164aac0 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:44.715378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41191928-bada-4441-9155-cf0f5164aac0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:44.715486 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41191928-bada-4441-9155-cf0f5164aac0 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:44.717048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:44.722433 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba551d-cea6-4b49-8689-8d5c57470206 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:44.722821 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba551d-cea6-4b49-8689-8d5c57470206 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:44.722999 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba551d-cea6-4b49-8689-8d5c57470206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:44.786260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-05ba551d-cea6-4b49-8689-8d5c57470206 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:44.787451 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-05ba551d-cea6-4b49-8689-8d5c57470206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:44.804825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:44.892584 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:22:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:44.928824 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41191928-bada-4441-9155-cf0f5164aac0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:44.930301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-41191928-bada-4441-9155-cf0f5164aac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:45.034261 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41191928-bada-4441-9155-cf0f5164aac0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:45.100453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:45.108756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00e16e-2e05-400b-9ca1-677b32e509ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:45.109403 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00e16e-2e05-400b-9ca1-677b32e509ee service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:45.109687 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00e16e-2e05-400b-9ca1-677b32e509ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:45.109960 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00e16e-2e05-400b-9ca1-677b32e509ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:45.111610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fb00e16e-2e05-400b-9ca1-677b32e509ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:45.218299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00e16e-2e05-400b-9ca1-677b32e509ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:45.219657 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00e16e-2e05-400b-9ca1-677b32e509ee service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:45.237935 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 743/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:47.288037 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f8664-436f-491a-bc56-b4378f68993a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:47.288609 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f8664-436f-491a-bc56-b4378f68993a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:47.288727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f8664-436f-491a-bc56-b4378f68993a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:47.290009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:22:47.295201 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3056ad-ec8b-416c-b6d1-7849e56e176e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:47.295592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3056ad-ec8b-416c-b6d1-7849e56e176e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:47.295774 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3056ad-ec8b-416c-b6d1-7849e56e176e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:47.359344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bb3056ad-ec8b-416c-b6d1-7849e56e176e None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:22:47.360573 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bb3056ad-ec8b-416c-b6d1-7849e56e176e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:47.407533 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:47.455293 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:22:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:22:47.493078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f8664-436f-491a-bc56-b4378f68993a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:22:47.495240 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-838f8664-436f-491a-bc56-b4378f68993a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:47.597180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f8664-436f-491a-bc56-b4378f68993a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:22:47.645700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:47.672590 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1b9f0-37b1-4976-86d7-860882e29ac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:22:47.673389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1b9f0-37b1-4976-86d7-860882e29ac5 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:22:47.673674 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1b9f0-37b1-4976-86d7-860882e29ac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:22:47.673950 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1b9f0-37b1-4976-86d7-860882e29ac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:22:47.675774 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9eb1b9f0-37b1-4976-86d7-860882e29ac5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:22:47.783238 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1b9f0-37b1-4976-86d7-860882e29ac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:22:47.784788 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1b9f0-37b1-4976-86d7-860882e29ac5 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:22:47.802943 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 726/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:22:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:22:57.453541 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=22 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:22:57.800828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=66 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:07.415738 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7922a5e-9653-4b99-84e1-e55cb026c53c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:07.416265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7922a5e-9653-4b99-84e1-e55cb026c53c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:07.416265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7922a5e-9653-4b99-84e1-e55cb026c53c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:07.417663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 745/1471] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:07.422955 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ee05e-7a0e-4395-a8f6-8d71adfe868d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:07.423368 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ee05e-7a0e-4395-a8f6-8d71adfe868d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:07.423520 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ee05e-7a0e-4395-a8f6-8d71adfe868d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:07.488493 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-760ee05e-7a0e-4395-a8f6-8d71adfe868d None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:07.489767 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-760ee05e-7a0e-4395-a8f6-8d71adfe868d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:07.591461 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 727/1472] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:23:07.640339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e697f0-2415-4adf-a090-20ca831bed13 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:07.640679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e697f0-2415-4adf-a090-20ca831bed13 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:07.640980 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e697f0-2415-4adf-a090-20ca831bed13 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:07.642351 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 746/1473] 10.222.0.33 () {54 vars in 961 bytes} [Mon Jul 18 10:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:07.647234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5176f54-35e3-48fd-85d1-994166e5bf80 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:07.647519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5176f54-35e3-48fd-85d1-994166e5bf80 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:07.647810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5176f54-35e3-48fd-85d1-994166e5bf80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:07.721045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e5176f54-35e3-48fd-85d1-994166e5bf80 None None] MFA Rules not processed for user `3cee4d23797e430e9eeddc394ed7f50e`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:07.722181 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e5176f54-35e3-48fd-85d1-994166e5bf80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:07.742951 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:07.797451 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 728/1474] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Jul 18 10:23:07 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:23:07.804945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e697f0-2415-4adf-a090-20ca831bed13 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:07.806473 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-47e697f0-2415-4adf-a090-20ca831bed13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:07.909973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e697f0-2415-4adf-a090-20ca831bed13 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:07.983395 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32071d5e-bf45-4137-9978-512bb02b6ca8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cee4d23797e430e9eeddc394ed7f50e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:07.983971 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32071d5e-bf45-4137-9978-512bb02b6ca8 service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:07.984393 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32071d5e-bf45-4137-9978-512bb02b6ca8 service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:07.984895 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32071d5e-bf45-4137-9978-512bb02b6ca8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:07.987853 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-32071d5e-bf45-4137-9978-512bb02b6ca8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:08.045367 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:08.100317 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32071d5e-bf45-4137-9978-512bb02b6ca8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:08.101784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32071d5e-bf45-4137-9978-512bb02b6ca8 service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:08.119910 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 747/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:23:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:23:10.241489 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbc272-6e4c-4456-8913-5bfe7d20b504 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:10.242035 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbc272-6e4c-4456-8913-5bfe7d20b504 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:10.242035 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbc272-6e4c-4456-8913-5bfe7d20b504 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:10.243514 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 729/1476] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:10.249334 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12611186-b61e-4d01-bce5-1b1d7c26202d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:10.249764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12611186-b61e-4d01-bce5-1b1d7c26202d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:10.249938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12611186-b61e-4d01-bce5-1b1d7c26202d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:10.313715 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-12611186-b61e-4d01-bce5-1b1d7c26202d None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:10.314930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-12611186-b61e-4d01-bce5-1b1d7c26202d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:10.409860 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 748/1477] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:23:10.432683 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbc272-6e4c-4456-8913-5bfe7d20b504 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:10.434702 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ffdbc272-6e4c-4456-8913-5bfe7d20b504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:10.557661 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbc272-6e4c-4456-8913-5bfe7d20b504 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:10.604847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:10.650103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861fb75-118f-4e98-89b9-95b366719d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:10.650759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861fb75-118f-4e98-89b9-95b366719d82 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:10.651041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861fb75-118f-4e98-89b9-95b366719d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:10.651316 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861fb75-118f-4e98-89b9-95b366719d82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:10.652041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861fb75-118f-4e98-89b9-95b366719d82 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:10.653853 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-4861fb75-118f-4e98-89b9-95b366719d82 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:23:10.653853 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 18 10:23:10.653853 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: Jul 18 10:23:10.654989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861fb75-118f-4e98-89b9-95b366719d82 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:10.673763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 730/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jul 18 10:23:10 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:23:12.693384 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650480b2-1754-4e90-b6a5-3d4994dd2285 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:12.693821 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650480b2-1754-4e90-b6a5-3d4994dd2285 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:12.693951 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650480b2-1754-4e90-b6a5-3d4994dd2285 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:12.695356 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 749/1479] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:12.701501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606b381-4588-4226-92bc-c6592b326e84 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:12.701942 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606b381-4588-4226-92bc-c6592b326e84 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:12.702083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606b381-4588-4226-92bc-c6592b326e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:12.767719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8606b381-4588-4226-92bc-c6592b326e84 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:12.768961 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8606b381-4588-4226-92bc-c6592b326e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:12.863187 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 731/1480] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:23:12.905399 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650480b2-1754-4e90-b6a5-3d4994dd2285 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:12.906925 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-650480b2-1754-4e90-b6a5-3d4994dd2285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:12.916675 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:13.052134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650480b2-1754-4e90-b6a5-3d4994dd2285 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:13.135985 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:13.172336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886d016-371e-417e-972b-c9cddaceb206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:13.173014 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886d016-371e-417e-972b-c9cddaceb206 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:13.173303 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886d016-371e-417e-972b-c9cddaceb206 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:13.173583 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886d016-371e-417e-972b-c9cddaceb206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:13.175247 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4886d016-371e-417e-972b-c9cddaceb206 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:13.270188 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886d016-371e-417e-972b-c9cddaceb206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:13.271810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886d016-371e-417e-972b-c9cddaceb206 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:13.289881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 750/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:23:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:23:15.466137 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec48827-23ac-4e18-babe-d99c8acb371b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:15.466666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec48827-23ac-4e18-babe-d99c8acb371b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:15.466666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec48827-23ac-4e18-babe-d99c8acb371b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:15.467995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 732/1482] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:15.473258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37305731-b522-4033-8cab-c4a980aa17aa None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:15.473702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37305731-b522-4033-8cab-c4a980aa17aa None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:15.473842 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37305731-b522-4033-8cab-c4a980aa17aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:15.538395 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-37305731-b522-4033-8cab-c4a980aa17aa None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:15.539597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-37305731-b522-4033-8cab-c4a980aa17aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:15.585695 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:15.636622 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 751/1483] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:23:15.673031 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec48827-23ac-4e18-babe-d99c8acb371b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:15.674559 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fec48827-23ac-4e18-babe-d99c8acb371b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:15.697438 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:15.776649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec48827-23ac-4e18-babe-d99c8acb371b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:15.850587 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3bf91-1c32-4973-b025-39ea24b57938 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:15.851235 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3bf91-1c32-4973-b025-39ea24b57938 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:15.851517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3bf91-1c32-4973-b025-39ea24b57938 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:15.851793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3bf91-1c32-4973-b025-39ea24b57938 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:15.853473 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b1e3bf91-1c32-4973-b025-39ea24b57938 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:15.926122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:15.960790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e3bf91-1c32-4973-b025-39ea24b57938 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:15.962222 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e3bf91-1c32-4973-b025-39ea24b57938 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:15.981329 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 733/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:23:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:23:18.065639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fdccc-c114-454a-b483-2fc2c8425675 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:18.066208 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fdccc-c114-454a-b483-2fc2c8425675 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:18.066208 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fdccc-c114-454a-b483-2fc2c8425675 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:18.067614 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 752/1485] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:18.073128 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f075c-f396-4151-9565-d2e0ba020c5e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:18.073562 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f075c-f396-4151-9565-d2e0ba020c5e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:18.073697 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f075c-f396-4151-9565-d2e0ba020c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:18.138832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b06f075c-f396-4151-9565-d2e0ba020c5e None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:18.140039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b06f075c-f396-4151-9565-d2e0ba020c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:18.237884 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 734/1486] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:23:18.276389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fdccc-c114-454a-b483-2fc2c8425675 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:18.278536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6e5fdccc-c114-454a-b483-2fc2c8425675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:18.388092 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fdccc-c114-454a-b483-2fc2c8425675 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:18.457455 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:18.459214 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ade34-4e00-4303-a805-92383943f5f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:18.459855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ade34-4e00-4303-a805-92383943f5f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:18.460370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ade34-4e00-4303-a805-92383943f5f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:18.460735 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ade34-4e00-4303-a805-92383943f5f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:18.462808 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-df5ade34-4e00-4303-a805-92383943f5f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:18.569704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ade34-4e00-4303-a805-92383943f5f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:18.571132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ade34-4e00-4303-a805-92383943f5f3 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:18.589580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 753/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:23:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:23:20.674302 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548c73e-9d27-4087-b917-8f998915753c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:20.674846 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548c73e-9d27-4087-b917-8f998915753c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:20.674846 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548c73e-9d27-4087-b917-8f998915753c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:20.676320 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 735/1488] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:20.681654 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545976b5-2bc0-4eeb-b721-1d370b1d117f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:20.682082 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545976b5-2bc0-4eeb-b721-1d370b1d117f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:20.682217 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545976b5-2bc0-4eeb-b721-1d370b1d117f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:20.746773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-545976b5-2bc0-4eeb-b721-1d370b1d117f None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:20.747989 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-545976b5-2bc0-4eeb-b721-1d370b1d117f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:20.800032 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:20.843830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 754/1489] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:23:20.884473 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548c73e-9d27-4087-b917-8f998915753c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:20.886067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e548c73e-9d27-4087-b917-8f998915753c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:20.923449 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:20.990193 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548c73e-9d27-4087-b917-8f998915753c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:21.060770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158f002-2d3a-4a45-b989-31946de06a32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:21.061413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158f002-2d3a-4a45-b989-31946de06a32 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:21.061695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158f002-2d3a-4a45-b989-31946de06a32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:21.061976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158f002-2d3a-4a45-b989-31946de06a32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:21.063602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e158f002-2d3a-4a45-b989-31946de06a32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:21.142930 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:21.172464 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158f002-2d3a-4a45-b989-31946de06a32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:21.173974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158f002-2d3a-4a45-b989-31946de06a32 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:21.192216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 736/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:23:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:23:23.401117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2a398-8bdf-4074-b39f-3236938616de None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:23.401712 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2a398-8bdf-4074-b39f-3236938616de None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:23.401712 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2a398-8bdf-4074-b39f-3236938616de None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:23.403624 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 755/1491] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jul 18 10:23:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:23:23.409013 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d05f9-f1b3-4159-aac5-32d4769f4d85 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:23.409445 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d05f9-f1b3-4159-aac5-32d4769f4d85 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:23.409550 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d05f9-f1b3-4159-aac5-32d4769f4d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:23.473055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d95d05f9-f1b3-4159-aac5-32d4769f4d85 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:23:23.474257 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d95d05f9-f1b3-4159-aac5-32d4769f4d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:23.572888 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 737/1492] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jul 18 10:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:23:23.612865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2a398-8bdf-4074-b39f-3236938616de None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:23:23.614407 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5ab2a398-8bdf-4074-b39f-3236938616de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:23.720793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2a398-8bdf-4074-b39f-3236938616de None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:23:23.792793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903c7b0-8db5-4a1c-b493-9c9747a74e5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:23:23.793502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903c7b0-8db5-4a1c-b493-9c9747a74e5f service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:23:23.793787 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903c7b0-8db5-4a1c-b493-9c9747a74e5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:23:23.794062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903c7b0-8db5-4a1c-b493-9c9747a74e5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:23:23.795697 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d903c7b0-8db5-4a1c-b493-9c9747a74e5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:23:23.804496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:23.903121 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903c7b0-8db5-4a1c-b493-9c9747a74e5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:23:23.904570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903c7b0-8db5-4a1c-b493-9c9747a74e5f service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:23:23.921986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 756/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:23:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:23:33.571044 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=82 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:23:33.919737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=50 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:56.095401 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9a3b3-a2ee-4523-9c2d-013cc3810137 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:56.096013 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9a3b3-a2ee-4523-9c2d-013cc3810137 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:56.096013 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9a3b3-a2ee-4523-9c2d-013cc3810137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:56.168143 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-21d9a3b3-a2ee-4523-9c2d-013cc3810137 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:56.169367 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-21d9a3b3-a2ee-4523-9c2d-013cc3810137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:56.265459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 738/1494] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:24:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:24:56.273631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3744df6-1800-4aef-9966-f0b9f4040961 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:56.274115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3744df6-1800-4aef-9966-f0b9f4040961 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:56.274221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3744df6-1800-4aef-9966-f0b9f4040961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:56.472588 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e3744df6-1800-4aef-9966-f0b9f4040961 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:56.473899 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e3744df6-1800-4aef-9966-f0b9f4040961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:56.568237 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 757/1495] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:24:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:24:56.573395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9a3b3-a2ee-4523-9c2d-013cc3810137 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:56.574605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-21d9a3b3-a2ee-4523-9c2d-013cc3810137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:56.586203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9a3b3-a2ee-4523-9c2d-013cc3810137 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:56.649097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:56.682784 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86810e3b-8411-49ce-94b0-3a8d90d6b28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:56.683448 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86810e3b-8411-49ce-94b0-3a8d90d6b28c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:56.683732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86810e3b-8411-49ce-94b0-3a8d90d6b28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:56.684027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86810e3b-8411-49ce-94b0-3a8d90d6b28c admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:56.684763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86810e3b-8411-49ce-94b0-3a8d90d6b28c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:24:56.685073 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86810e3b-8411-49ce-94b0-3a8d90d6b28c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:56.686093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-86810e3b-8411-49ce-94b0-3a8d90d6b28c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:24:56.686093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 18 10:24:56.686093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: Jul 18 10:24:56.686990 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86810e3b-8411-49ce-94b0-3a8d90d6b28c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:56.698623 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 739/1496] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:24:56.706848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3744df6-1800-4aef-9966-f0b9f4040961 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:56.716042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3744df6-1800-4aef-9966-f0b9f4040961 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:56.778240 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:56.811228 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931832a-259e-4b7c-8ae7-62fddc07985c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:56.811881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931832a-259e-4b7c-8ae7-62fddc07985c admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:56.812165 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931832a-259e-4b7c-8ae7-62fddc07985c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:56.812459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931832a-259e-4b7c-8ae7-62fddc07985c admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:56.813165 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b931832a-259e-4b7c-8ae7-62fddc07985c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:56.814902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-b931832a-259e-4b7c-8ae7-62fddc07985c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:24:56.814902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 18 10:24:56.814902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: Jul 18 10:24:56.816097 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b931832a-259e-4b7c-8ae7-62fddc07985c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:56.871084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 758/1497] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:24:56 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:56.876216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86810e3b-8411-49ce-94b0-3a8d90d6b28c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:56.877818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-86810e3b-8411-49ce-94b0-3a8d90d6b28c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:56.911038 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86810e3b-8411-49ce-94b0-3a8d90d6b28c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:57.002981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eda5ad-5438-468a-b39d-f794d559d399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:57.003573 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eda5ad-5438-468a-b39d-f794d559d399 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:57.003856 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eda5ad-5438-468a-b39d-f794d559d399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:57.004138 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eda5ad-5438-468a-b39d-f794d559d399 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:57.004867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eda5ad-5438-468a-b39d-f794d559d399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:57.005871 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-45eda5ad-5438-468a-b39d-f794d559d399 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:24:57.005871 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 18 10:24:57.005871 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: Jul 18 10:24:57.007088 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eda5ad-5438-468a-b39d-f794d559d399 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:57.046312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 740/1498] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:24:56 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:24:57.050331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931832a-259e-4b7c-8ae7-62fddc07985c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:57.051857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b931832a-259e-4b7c-8ae7-62fddc07985c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:57.112905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931832a-259e-4b7c-8ae7-62fddc07985c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:57.194374 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:57.205697 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebea03-08ec-40e8-b2c7-505fb5e10241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:57.206357 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebea03-08ec-40e8-b2c7-505fb5e10241 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:57.206637 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebea03-08ec-40e8-b2c7-505fb5e10241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:57.206911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebea03-08ec-40e8-b2c7-505fb5e10241 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:57.207627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deebea03-08ec-40e8-b2c7-505fb5e10241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:57.208682 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-deebea03-08ec-40e8-b2c7-505fb5e10241 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:24:57.208682 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 18 10:24:57.208682 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: Jul 18 10:24:57.209559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deebea03-08ec-40e8-b2c7-505fb5e10241 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:57.222911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 759/1499] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:24:57.227008 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eda5ad-5438-468a-b39d-f794d559d399 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:57.229155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-45eda5ad-5438-468a-b39d-f794d559d399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:57.240867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eda5ad-5438-468a-b39d-f794d559d399 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:57.265928 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:57.336399 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64fdc7-2cf7-4346-930a-badf5929b308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:57.337599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64fdc7-2cf7-4346-930a-badf5929b308 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:57.337882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64fdc7-2cf7-4346-930a-badf5929b308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:57.338160 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64fdc7-2cf7-4346-930a-badf5929b308 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:57.338885 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64fdc7-2cf7-4346-930a-badf5929b308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:57.339907 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-7f64fdc7-2cf7-4346-930a-badf5929b308 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:24:57.339907 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 18 10:24:57.339907 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: Jul 18 10:24:57.340824 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64fdc7-2cf7-4346-930a-badf5929b308 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:57.353663 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 741/1500] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:24:57.358101 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebea03-08ec-40e8-b2c7-505fb5e10241 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:57.359610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-deebea03-08ec-40e8-b2c7-505fb5e10241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:57.391936 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebea03-08ec-40e8-b2c7-505fb5e10241 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:57.484878 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a87740-1284-4ddf-9621-df663a4ef051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:57.485767 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a87740-1284-4ddf-9621-df663a4ef051 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:57.486049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a87740-1284-4ddf-9621-df663a4ef051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:57.486321 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a87740-1284-4ddf-9621-df663a4ef051 admin admin] PATH_INFO: `/v3/projects/50102248ec904ea9be9b1483ba1cdc02/users/bb9372cc1fea464798d4f8675114ade4/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:57.508404 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a87740-1284-4ddf-9621-df663a4ef051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50102248ec904ea9be9b1483ba1cdc02, user_id=bb9372cc1fea464798d4f8675114ade4, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:57.509759 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-a3a87740-1284-4ddf-9621-df663a4ef051 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:24:57.509759 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 18 10:24:57.509759 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: Jul 18 10:24:57.511656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a87740-1284-4ddf-9621-df663a4ef051 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:57.536661 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 760/1501] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:24:57 2022] PUT /identity/v3/projects/50102248ec904ea9be9b1483ba1cdc02/users/bb9372cc1fea464798d4f8675114ade4/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:57.549201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43333ed8-4591-4a5a-b19c-82c0a54b2aff None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:57.549500 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43333ed8-4591-4a5a-b19c-82c0a54b2aff None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:57.549779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43333ed8-4591-4a5a-b19c-82c0a54b2aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:57.605931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-43333ed8-4591-4a5a-b19c-82c0a54b2aff None None] MFA Rules not processed for user `bb9372cc1fea464798d4f8675114ade4`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:57.607154 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-43333ed8-4591-4a5a-b19c-82c0a54b2aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:57.682668 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 742/1502] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:24:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:24:57.695753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d53f5f-8e92-4964-9ed3-eb4549b4dd1c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:57.696044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d53f5f-8e92-4964-9ed3-eb4549b4dd1c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:57.696319 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d53f5f-8e92-4964-9ed3-eb4549b4dd1c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:57.698233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 761/1503] 10.222.0.33 () {54 vars in 964 bytes} [Mon Jul 18 10:24:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:24:57.703006 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0daf15d-a02d-43d7-97fa-5918d7b0b15c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:57.703321 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0daf15d-a02d-43d7-97fa-5918d7b0b15c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:57.703603 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0daf15d-a02d-43d7-97fa-5918d7b0b15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:57.733450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:57.777439 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f0daf15d-a02d-43d7-97fa-5918d7b0b15c None None] MFA Rules not processed for user `20e29c37dc924514a882f936f5936ae1`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:57.778639 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f0daf15d-a02d-43d7-97fa-5918d7b0b15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:57.852720 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 743/1504] 10.222.0.33 () {58 vars in 1061 bytes} [Mon Jul 18 10:24:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:24:57.860529 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d53f5f-8e92-4964-9ed3-eb4549b4dd1c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:57.862036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c8d53f5f-8e92-4964-9ed3-eb4549b4dd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:57.910565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:57.962758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d53f5f-8e92-4964-9ed3-eb4549b4dd1c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:58.034633 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc18f6-6003-461a-ad89-7e895ae3174f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:58.035275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc18f6-6003-461a-ad89-7e895ae3174f service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:58.035558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc18f6-6003-461a-ad89-7e895ae3174f service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:58.035832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc18f6-6003-461a-ad89-7e895ae3174f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:58.037502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6bbc18f6-6003-461a-ad89-7e895ae3174f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:58.133618 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc18f6-6003-461a-ad89-7e895ae3174f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:58.135174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbc18f6-6003-461a-ad89-7e895ae3174f service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:58.139281 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:58.154172 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 762/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:24:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:24:58.196311 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefd3b9-72f5-4a65-9863-9293d9b41fb2 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:58.196663 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefd3b9-72f5-4a65-9863-9293d9b41fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:58.196973 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefd3b9-72f5-4a65-9863-9293d9b41fb2 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:58.198879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 744/1506] 10.222.0.33 () {52 vars in 829 bytes} [Mon Jul 18 10:24:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:24:58.202551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc18f6-6003-461a-ad89-7e895ae3174f service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:58.203793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6bbc18f6-6003-461a-ad89-7e895ae3174f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:58.302663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc18f6-6003-461a-ad89-7e895ae3174f service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:58.399470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fbe2b1-87b4-485f-b5ce-08285bb5be38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:58.400253 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fbe2b1-87b4-485f-b5ce-08285bb5be38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:58.400560 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fbe2b1-87b4-485f-b5ce-08285bb5be38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:58.400868 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fbe2b1-87b4-485f-b5ce-08285bb5be38 admin admin] PATH_INFO: `/v3/users/bb9372cc1fea464798d4f8675114ade4` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:58.412897 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fbe2b1-87b4-485f-b5ce-08285bb5be38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb9372cc1fea464798d4f8675114ade4)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:58.413926 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-c5fbe2b1-87b4-485f-b5ce-08285bb5be38 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:24:58.413926 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 18 10:24:58.413926 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: Jul 18 10:24:58.415146 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fbe2b1-87b4-485f-b5ce-08285bb5be38 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:58.425114 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:58.477901 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c5fbe2b1-87b4-485f-b5ce-08285bb5be38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb9372cc1fea464798d4f8675114ade4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:24:58.496142 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c5fbe2b1-87b4-485f-b5ce-08285bb5be38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb9372cc1fea464798d4f8675114ade4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:24:58.496734 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c5fbe2b1-87b4-485f-b5ce-08285bb5be38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb9372cc1fea464798d4f8675114ade4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:24:58.497170 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c5fbe2b1-87b4-485f-b5ce-08285bb5be38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb9372cc1fea464798d4f8675114ade4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:24:58.507026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c5fbe2b1-87b4-485f-b5ce-08285bb5be38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb9372cc1fea464798d4f8675114ade4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:24:58.527273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 763/1507] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:24:58 2022] DELETE /identity/v3/users/bb9372cc1fea464798d4f8675114ade4 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:24:58.531757 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe7466-c449-4bb7-9809-dd6ebcc715fc None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:58.532242 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe7466-c449-4bb7-9809-dd6ebcc715fc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:58.532353 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe7466-c449-4bb7-9809-dd6ebcc715fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:58.586977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4ebe7466-c449-4bb7-9809-dd6ebcc715fc None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:24:58.588205 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4ebe7466-c449-4bb7-9809-dd6ebcc715fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:58.663434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:58.676387 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 745/1508] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:24:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:24:58.686792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fbe2b1-87b4-485f-b5ce-08285bb5be38 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:58.688006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c5fbe2b1-87b4-485f-b5ce-08285bb5be38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:58.786894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fbe2b1-87b4-485f-b5ce-08285bb5be38 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:58.843372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:24:58.864674 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d33d6-5755-46ab-b145-62ad6ff70f46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:58.865344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d33d6-5755-46ab-b145-62ad6ff70f46 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:58.865626 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d33d6-5755-46ab-b145-62ad6ff70f46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:58.865901 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d33d6-5755-46ab-b145-62ad6ff70f46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:58.867521 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-677d33d6-5755-46ab-b145-62ad6ff70f46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:58.976992 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d33d6-5755-46ab-b145-62ad6ff70f46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:58.978350 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d33d6-5755-46ab-b145-62ad6ff70f46 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:58.997443 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 764/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:24:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:24:59.326956 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe7466-c449-4bb7-9809-dd6ebcc715fc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:24:59.328240 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4ebe7466-c449-4bb7-9809-dd6ebcc715fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:24:59.340152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe7466-c449-4bb7-9809-dd6ebcc715fc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:24:59.446385 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825fb7dd-ef14-448a-b988-1b44a09e1666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:24:59.447245 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fb7dd-ef14-448a-b988-1b44a09e1666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:24:59.448062 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fb7dd-ef14-448a-b988-1b44a09e1666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:24:59.448322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825fb7dd-ef14-448a-b988-1b44a09e1666 admin admin] PATH_INFO: `/v3/projects/50102248ec904ea9be9b1483ba1cdc02` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:24:59.458267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fb7dd-ef14-448a-b988-1b44a09e1666 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50102248ec904ea9be9b1483ba1cdc02)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:24:59.459301 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-825fb7dd-ef14-448a-b988-1b44a09e1666 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:24:59.459301 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 18 10:24:59.459301 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: Jul 18 10:24:59.460520 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825fb7dd-ef14-448a-b988-1b44a09e1666 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:24:59.518217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-825fb7dd-ef14-448a-b988-1b44a09e1666 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50102248ec904ea9be9b1483ba1cdc02', 'request_id': 'req-825fb7dd-ef14-448a-b988-1b44a09e1666'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:24:59.524109 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-825fb7dd-ef14-448a-b988-1b44a09e1666 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50102248ec904ea9be9b1483ba1cdc02', 'request_id': 'req-825fb7dd-ef14-448a-b988-1b44a09e1666'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:24:59.542336 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-825fb7dd-ef14-448a-b988-1b44a09e1666 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50102248ec904ea9be9b1483ba1cdc02', 'request_id': 'req-825fb7dd-ef14-448a-b988-1b44a09e1666'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:24:59.568605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-825fb7dd-ef14-448a-b988-1b44a09e1666 admin admin] The token cache is being invalidate because project 50102248ec904ea9be9b1483ba1cdc02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:24:59.568897 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-825fb7dd-ef14-448a-b988-1b44a09e1666 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:24:59.569410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-825fb7dd-ef14-448a-b988-1b44a09e1666 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:24:59.570773 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 746/1510] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:24:59 2022] DELETE /identity/v3/projects/50102248ec904ea9be9b1483ba1cdc02 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:25:08.995124 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=62 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:25:09.566671 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:49.816309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648a9c7-9e88-4137-bca4-8196e944df37 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:49.816944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648a9c7-9e88-4137-bca4-8196e944df37 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:49.816944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648a9c7-9e88-4137-bca4-8196e944df37 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:49.818294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:49.823678 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594235fe-8195-46f7-a27a-8f8072a38b8d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:49.824122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594235fe-8195-46f7-a27a-8f8072a38b8d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:49.824233 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594235fe-8195-46f7-a27a-8f8072a38b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:49.901529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-594235fe-8195-46f7-a27a-8f8072a38b8d None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:49.902735 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-594235fe-8195-46f7-a27a-8f8072a38b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:50.012039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:26:50.173668 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648a9c7-9e88-4137-bca4-8196e944df37 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:50.175808 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f648a9c7-9e88-4137-bca4-8196e944df37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:50.296787 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648a9c7-9e88-4137-bca4-8196e944df37 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:50.302303 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847137b3-ff94-47ba-b8e9-56630493c177 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:50.302954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847137b3-ff94-47ba-b8e9-56630493c177 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:50.303243 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847137b3-ff94-47ba-b8e9-56630493c177 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:50.303553 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847137b3-ff94-47ba-b8e9-56630493c177 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:50.311922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847137b3-ff94-47ba-b8e9-56630493c177 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:50.313868 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847137b3-ff94-47ba-b8e9-56630493c177 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:50.324327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 766/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:26:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:26:51.187985 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cddeeb-75c8-4997-8c20-a9907bd2ad84 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:51.188521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cddeeb-75c8-4997-8c20-a9907bd2ad84 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:51.188699 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cddeeb-75c8-4997-8c20-a9907bd2ad84 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:51.189988 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:51.194794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7798827-273a-470a-b34a-04f07f8493f2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:51.195243 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7798827-273a-470a-b34a-04f07f8493f2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:51.195348 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7798827-273a-470a-b34a-04f07f8493f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:51.213124 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d7798827-273a-470a-b34a-04f07f8493f2 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:51.214325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d7798827-273a-470a-b34a-04f07f8493f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:51.222231 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:26:51.374756 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cddeeb-75c8-4997-8c20-a9907bd2ad84 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:51.388806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cddeeb-75c8-4997-8c20-a9907bd2ad84 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:51.394138 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b68ffb-c33d-4dff-baca-f6052d021113 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:51.394766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b68ffb-c33d-4dff-baca-f6052d021113 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:51.395049 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b68ffb-c33d-4dff-baca-f6052d021113 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:51.395322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b68ffb-c33d-4dff-baca-f6052d021113 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:51.405138 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b68ffb-c33d-4dff-baca-f6052d021113 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:51.406798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b68ffb-c33d-4dff-baca-f6052d021113 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:51.417060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 749/1516] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:26:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:26:51.492246 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9cd6d-4578-4f65-bfdb-a65445727910 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:51.492716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9cd6d-4578-4f65-bfdb-a65445727910 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:51.493012 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9cd6d-4578-4f65-bfdb-a65445727910 None None] PATH_INFO: `/v3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:51.494392 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 768/1517] 10.222.0.30 () {54 vars in 925 bytes} [Mon Jul 18 10:26:51 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:26:51.518354 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e2498-9726-4a86-856f-4d645f420cdf None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:51.518646 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e2498-9726-4a86-856f-4d645f420cdf None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:51.518923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e2498-9726-4a86-856f-4d645f420cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:51.576773 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d71e2498-9726-4a86-856f-4d645f420cdf None None] MFA Rules not processed for user `d07165f554a445169b2c3f0d173c0bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:51.577964 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d71e2498-9726-4a86-856f-4d645f420cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:51.594873 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:26:51.650192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 750/1518] 10.222.0.30 () {58 vars in 1015 bytes} [Mon Jul 18 10:26:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:26:55.498998 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087db16-4c1c-4840-88a1-04779ec5fb31 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:55.499567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087db16-4c1c-4840-88a1-04779ec5fb31 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:55.499567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087db16-4c1c-4840-88a1-04779ec5fb31 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:55.501005 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 769/1519] 10.222.0.33 () {54 vars in 961 bytes} [Mon Jul 18 10:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:26:55.506061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500d1ae-c21f-4ea7-833a-dac340168713 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:55.506460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500d1ae-c21f-4ea7-833a-dac340168713 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:55.506621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500d1ae-c21f-4ea7-833a-dac340168713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:55.524387 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6500d1ae-c21f-4ea7-833a-dac340168713 None None] MFA Rules not processed for user `d07165f554a445169b2c3f0d173c0bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:26:55.525592 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6500d1ae-c21f-4ea7-833a-dac340168713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:26:55.534290 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 751/1520] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Jul 18 10:26:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:26:55.542624 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087db16-4c1c-4840-88a1-04779ec5fb31 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:26:55.556470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087db16-4c1c-4840-88a1-04779ec5fb31 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:26:55.561437 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8097838-5536-41d5-bcf6-f312cb94398a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd07165f554a445169b2c3f0d173c0bb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:26:55.562062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8097838-5536-41d5-bcf6-f312cb94398a service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:26:55.562346 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8097838-5536-41d5-bcf6-f312cb94398a service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:26:55.562622 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8097838-5536-41d5-bcf6-f312cb94398a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:26:55.570877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8097838-5536-41d5-bcf6-f312cb94398a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:26:55.572449 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8097838-5536-41d5-bcf6-f312cb94398a service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:26:55.581964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 770/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:26:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:27:02.624779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4198259-63d7-4aa5-b928-4d4fce9c1005 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:02.625403 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4198259-63d7-4aa5-b928-4d4fce9c1005 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:02.625403 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4198259-63d7-4aa5-b928-4d4fce9c1005 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:02.626712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:27:02.632344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c06ef-76c8-4df6-bf34-73a0d8c1060a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:02.632797 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c06ef-76c8-4df6-bf34-73a0d8c1060a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:02.632946 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c06ef-76c8-4df6-bf34-73a0d8c1060a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:02.650459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-594c06ef-76c8-4df6-bf34-73a0d8c1060a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:27:02.651654 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-594c06ef-76c8-4df6-bf34-73a0d8c1060a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:27:02.659438 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:27:02.814181 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4198259-63d7-4aa5-b928-4d4fce9c1005 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:02.829477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4198259-63d7-4aa5-b928-4d4fce9c1005 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:02.835114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94844ebf-1734-4f60-9a40-d0f2e1cfd94e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:02.835745 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94844ebf-1734-4f60-9a40-d0f2e1cfd94e service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:02.836027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94844ebf-1734-4f60-9a40-d0f2e1cfd94e service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:02.836310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94844ebf-1734-4f60-9a40-d0f2e1cfd94e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:02.844374 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94844ebf-1734-4f60-9a40-d0f2e1cfd94e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:02.845973 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94844ebf-1734-4f60-9a40-d0f2e1cfd94e service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:02.855631 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 753/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:27:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:27:03.690602 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bfe1a-b0d1-476e-a45c-46b1de44ac2e None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:03.691168 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bfe1a-b0d1-476e-a45c-46b1de44ac2e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:03.691168 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bfe1a-b0d1-476e-a45c-46b1de44ac2e None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:03.692574 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 10:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:27:03.697898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77859dd-08f1-42cd-ac99-e138a6e750a1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:03.698290 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77859dd-08f1-42cd-ac99-e138a6e750a1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:03.698461 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77859dd-08f1-42cd-ac99-e138a6e750a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:03.717211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d77859dd-08f1-42cd-ac99-e138a6e750a1 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:27:03.718424 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d77859dd-08f1-42cd-ac99-e138a6e750a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:27:03.726370 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 10:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:27:03.878031 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bfe1a-b0d1-476e-a45c-46b1de44ac2e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:03.892575 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bfe1a-b0d1-476e-a45c-46b1de44ac2e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:03.898357 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff95703-10cd-4043-a064-ad4488fc0674 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:03.898984 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff95703-10cd-4043-a064-ad4488fc0674 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:03.899265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff95703-10cd-4043-a064-ad4488fc0674 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:03.899537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff95703-10cd-4043-a064-ad4488fc0674 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:03.907412 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff95703-10cd-4043-a064-ad4488fc0674 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:03.909020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff95703-10cd-4043-a064-ad4488fc0674 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:03.918188 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 773/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:27:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:27:07.927869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0f054-4c4e-47f9-9b02-d9b3cd4e3cfc None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:07.928984 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0f054-4c4e-47f9-9b02-d9b3cd4e3cfc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:07.929108 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0f054-4c4e-47f9-9b02-d9b3cd4e3cfc None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:07.930472 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:27:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:27:07.935761 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704347a9-9191-4136-9e8d-6996363cce2e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:07.936212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704347a9-9191-4136-9e8d-6996363cce2e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:07.936312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704347a9-9191-4136-9e8d-6996363cce2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:07.954366 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-704347a9-9191-4136-9e8d-6996363cce2e None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:27:07.955544 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-704347a9-9191-4136-9e8d-6996363cce2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:27:07.963235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:27:08.002103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0f054-4c4e-47f9-9b02-d9b3cd4e3cfc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:08.003756 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e6a0f054-4c4e-47f9-9b02-d9b3cd4e3cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:27:08.085961 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0f054-4c4e-47f9-9b02-d9b3cd4e3cfc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:08.089660 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65eeb6e-f90f-4fcb-abbd-5a7c1877cf88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:08.090313 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65eeb6e-f90f-4fcb-abbd-5a7c1877cf88 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:08.090602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65eeb6e-f90f-4fcb-abbd-5a7c1877cf88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:08.090886 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65eeb6e-f90f-4fcb-abbd-5a7c1877cf88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:08.099835 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65eeb6e-f90f-4fcb-abbd-5a7c1877cf88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:08.101307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65eeb6e-f90f-4fcb-abbd-5a7c1877cf88 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:08.111018 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 756/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:27:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:27:10.106295 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220003a-2a74-4a7a-8fe9-e4731ec7e989 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:10.106818 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220003a-2a74-4a7a-8fe9-e4731ec7e989 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:10.106818 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220003a-2a74-4a7a-8fe9-e4731ec7e989 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:10.108151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:27:10.113381 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9470d-149f-426d-aa72-38cf5c152f6a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:10.113809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9470d-149f-426d-aa72-38cf5c152f6a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:10.113949 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9470d-149f-426d-aa72-38cf5c152f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:10.131616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-90b9470d-149f-426d-aa72-38cf5c152f6a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:27:10.132843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-90b9470d-149f-426d-aa72-38cf5c152f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:27:10.140757 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:27:10.177618 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220003a-2a74-4a7a-8fe9-e4731ec7e989 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:10.188651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220003a-2a74-4a7a-8fe9-e4731ec7e989 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:10.192170 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb619e-fedf-4934-b67e-55955fdfb04a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:10.192847 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb619e-fedf-4934-b67e-55955fdfb04a service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:10.193137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb619e-fedf-4934-b67e-55955fdfb04a service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:10.193419 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb619e-fedf-4934-b67e-55955fdfb04a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:10.202706 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb619e-fedf-4934-b67e-55955fdfb04a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:10.204238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb619e-fedf-4934-b67e-55955fdfb04a service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:10.214172 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 776/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:27:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:27:13.767790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617532c4-1be3-46af-9c9f-043ad95bc6ee None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:13.768361 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617532c4-1be3-46af-9c9f-043ad95bc6ee None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:13.768361 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617532c4-1be3-46af-9c9f-043ad95bc6ee None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:13.769843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:27:13.775510 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3b47b-af7c-4a56-aa32-961671c62d2b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:13.775961 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3b47b-af7c-4a56-aa32-961671c62d2b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:13.776067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3b47b-af7c-4a56-aa32-961671c62d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:13.794606 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f9b3b47b-af7c-4a56-aa32-961671c62d2b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:27:13.795801 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f9b3b47b-af7c-4a56-aa32-961671c62d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:27:13.803664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:27:14.110947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617532c4-1be3-46af-9c9f-043ad95bc6ee None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:14.112634 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-617532c4-1be3-46af-9c9f-043ad95bc6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:27:14.176734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:27:14.190413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617532c4-1be3-46af-9c9f-043ad95bc6ee None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:14.194507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2ee76-63af-4596-9a46-fc9a5477cfd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:14.195153 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2ee76-63af-4596-9a46-fc9a5477cfd2 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:14.195439 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2ee76-63af-4596-9a46-fc9a5477cfd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:14.195714 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2ee76-63af-4596-9a46-fc9a5477cfd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:14.204616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2ee76-63af-4596-9a46-fc9a5477cfd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:14.206052 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2ee76-63af-4596-9a46-fc9a5477cfd2 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:14.215917 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 759/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:27:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:27:16.233006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca648db7-b141-4bdd-823f-e18b3835ac51 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:16.233529 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca648db7-b141-4bdd-823f-e18b3835ac51 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:16.233634 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca648db7-b141-4bdd-823f-e18b3835ac51 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:16.234955 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 10:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:27:16.240123 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c1041-7f5e-4533-9a4e-af25c7c7ea66 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:16.240532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c1041-7f5e-4533-9a4e-af25c7c7ea66 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:16.240714 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c1041-7f5e-4533-9a4e-af25c7c7ea66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:16.258443 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-765c1041-7f5e-4533-9a4e-af25c7c7ea66 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:27:16.259626 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-765c1041-7f5e-4533-9a4e-af25c7c7ea66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:27:16.267507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 10:27:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:27:16.567259 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca648db7-b141-4bdd-823f-e18b3835ac51 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:27:16.569506 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ca648db7-b141-4bdd-823f-e18b3835ac51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:27:16.582161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca648db7-b141-4bdd-823f-e18b3835ac51 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:27:16.586289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a135a42-5a4f-4809-8528-31df1a27e75c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:27:16.586923 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a135a42-5a4f-4809-8528-31df1a27e75c service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:27:16.587209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a135a42-5a4f-4809-8528-31df1a27e75c service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:27:16.587488 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a135a42-5a4f-4809-8528-31df1a27e75c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:27:16.596291 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a135a42-5a4f-4809-8528-31df1a27e75c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:27:16.597748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a135a42-5a4f-4809-8528-31df1a27e75c service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:27:16.607293 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 779/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:27:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:27:26.253432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=11 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:27:26.602964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:48.332158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8323b-ffb9-4152-aa51-46c638723316 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.332918 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8323b-ffb9-4152-aa51-46c638723316 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.332918 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8323b-ffb9-4152-aa51-46c638723316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.368492 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f5f8323b-ffb9-4152-aa51-46c638723316 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:48.369736 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f5f8323b-ffb9-4152-aa51-46c638723316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:48.384482 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 761/1540] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:48.393093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf21b41-9741-4033-a547-2370768ad60b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.393685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf21b41-9741-4033-a547-2370768ad60b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.393685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf21b41-9741-4033-a547-2370768ad60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.423090 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9bf21b41-9741-4033-a547-2370768ad60b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:48.424430 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9bf21b41-9741-4033-a547-2370768ad60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:48.438389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 780/1541] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:48.445605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8323b-ffb9-4152-aa51-46c638723316 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:48.460529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8323b-ffb9-4152-aa51-46c638723316 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:48.475825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a2904-5584-41b8-9224-07deb6a6dafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:48.476938 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a2904-5584-41b8-9224-07deb6a6dafa admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.477430 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a2904-5584-41b8-9224-07deb6a6dafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.477933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a2904-5584-41b8-9224-07deb6a6dafa admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.478877 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a2904-5584-41b8-9224-07deb6a6dafa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:48.479657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a2904-5584-41b8-9224-07deb6a6dafa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:48.481557 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a2904-5584-41b8-9224-07deb6a6dafa admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:48.517438 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 762/1542] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:48.525604 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf21b41-9741-4033-a547-2370768ad60b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:48.541842 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf21b41-9741-4033-a547-2370768ad60b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:48.544807 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fdd8f-a910-445c-893c-e8c6ca2b2c4a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.545349 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fdd8f-a910-445c-893c-e8c6ca2b2c4a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.545846 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fdd8f-a910-445c-893c-e8c6ca2b2c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.581395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f26fdd8f-a910-445c-893c-e8c6ca2b2c4a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:48.582980 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f26fdd8f-a910-445c-893c-e8c6ca2b2c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:48.585131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b852a65-b3a1-4af1-b60e-0e7ec9e14982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:48.588535 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b852a65-b3a1-4af1-b60e-0e7ec9e14982 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.588535 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b852a65-b3a1-4af1-b60e-0e7ec9e14982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.588535 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b852a65-b3a1-4af1-b60e-0e7ec9e14982 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.588535 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b852a65-b3a1-4af1-b60e-0e7ec9e14982 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:48.589264 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b852a65-b3a1-4af1-b60e-0e7ec9e14982 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:48.601963 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 763/1543] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:48.603119 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdf567-8bfc-4fda-afbf-d79ce6b6b65e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.603414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdf567-8bfc-4fda-afbf-d79ce6b6b65e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.603692 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdf567-8bfc-4fda-afbf-d79ce6b6b65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.641385 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-47bdf567-8bfc-4fda-afbf-d79ce6b6b65e None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:48.642670 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-47bdf567-8bfc-4fda-afbf-d79ce6b6b65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:48.655869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 764/1544] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:48.667779 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 781/1545] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:28:48 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:48.669470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a4199-5e27-48d5-88f7-196dbf691e45 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.669767 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a4199-5e27-48d5-88f7-196dbf691e45 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.670050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a4199-5e27-48d5-88f7-196dbf691e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.672060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375b932-e91d-446e-958a-1ac17eb510cd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.672348 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375b932-e91d-446e-958a-1ac17eb510cd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.672656 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375b932-e91d-446e-958a-1ac17eb510cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.706510 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b00a4199-5e27-48d5-88f7-196dbf691e45 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:48.708965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b00a4199-5e27-48d5-88f7-196dbf691e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:48.720646 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1375b932-e91d-446e-958a-1ac17eb510cd None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:48.722192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1375b932-e91d-446e-958a-1ac17eb510cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:48.789685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 782/1546] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:48.790832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d6f72-1c7e-4f06-8154-2258ab67c137 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.791125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d6f72-1c7e-4f06-8154-2258ab67c137 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.791407 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d6f72-1c7e-4f06-8154-2258ab67c137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.799453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 765/1547] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:48.801505 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375b932-e91d-446e-958a-1ac17eb510cd None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:48.813248 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375b932-e91d-446e-958a-1ac17eb510cd None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:48.815456 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-866d6f72-1c7e-4f06-8154-2258ab67c137 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:48.816704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-866d6f72-1c7e-4f06-8154-2258ab67c137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:48.818063 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d56d7-a0c6-465b-b517-1371677f6de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:48.818728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d56d7-a0c6-465b-b517-1371677f6de4 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.819012 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d56d7-a0c6-465b-b517-1371677f6de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.819283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d56d7-a0c6-465b-b517-1371677f6de4 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.819962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d56d7-a0c6-465b-b517-1371677f6de4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:48.821924 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d56d7-a0c6-465b-b517-1371677f6de4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:48.824723 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 783/1548] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:48.825952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8410cfa-d651-42ee-8771-61545a1e6fb3 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.826246 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8410cfa-d651-42ee-8771-61545a1e6fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.826525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8410cfa-d651-42ee-8771-61545a1e6fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.843940 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 766/1549] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:28:48 2022] POST /identity/v3/users => generated 286 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:48.845248 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08cd4a-10a6-4fb4-8e4a-28875842be47 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.845669 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08cd4a-10a6-4fb4-8e4a-28875842be47 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.845960 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08cd4a-10a6-4fb4-8e4a-28875842be47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.847836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f8410cfa-d651-42ee-8771-61545a1e6fb3 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:48.849115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f8410cfa-d651-42ee-8771-61545a1e6fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:48.857872 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 784/1550] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:48.859097 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c3148-7e74-445f-ad92-de016c30f93a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.859391 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c3148-7e74-445f-ad92-de016c30f93a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.859667 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c3148-7e74-445f-ad92-de016c30f93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.864804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9a08cd4a-10a6-4fb4-8e4a-28875842be47 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:48.866272 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9a08cd4a-10a6-4fb4-8e4a-28875842be47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:48.875750 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 767/1551] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:48.879435 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08cd4a-10a6-4fb4-8e4a-28875842be47 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:48.882473 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e88c3148-7e74-445f-ad92-de016c30f93a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:48.883750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e88c3148-7e74-445f-ad92-de016c30f93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:48.886496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08cd4a-10a6-4fb4-8e4a-28875842be47 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:48.891415 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb280990-a288-4618-ac3f-e561a5d38f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:48.892292 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb280990-a288-4618-ac3f-e561a5d38f96 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.892425 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 785/1552] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:48.892549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb280990-a288-4618-ac3f-e561a5d38f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.892739 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb280990-a288-4618-ac3f-e561a5d38f96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.893356 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d5efb-cab8-4a25-8460-b0bde3cf6f7d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.893543 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb280990-a288-4618-ac3f-e561a5d38f96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:48.893657 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d5efb-cab8-4a25-8460-b0bde3cf6f7d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.893835 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb280990-a288-4618-ac3f-e561a5d38f96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:48.893947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d5efb-cab8-4a25-8460-b0bde3cf6f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.895583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb280990-a288-4618-ac3f-e561a5d38f96 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:48.911295 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 768/1553] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:48.913204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e382d2-56e5-4e51-ae92-73c02936f92b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.913494 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e382d2-56e5-4e51-ae92-73c02936f92b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.913771 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e382d2-56e5-4e51-ae92-73c02936f92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.914305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-054d5efb-cab8-4a25-8460-b0bde3cf6f7d None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:48.915718 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-054d5efb-cab8-4a25-8460-b0bde3cf6f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:48.924306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 786/1554] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:48.925564 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bb87f-8acd-4fd5-819d-8f08a1be69d8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.925938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bb87f-8acd-4fd5-819d-8f08a1be69d8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.926220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bb87f-8acd-4fd5-819d-8f08a1be69d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.932228 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f6e382d2-56e5-4e51-ae92-73c02936f92b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:48.933453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f6e382d2-56e5-4e51-ae92-73c02936f92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:48.941646 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 769/1555] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:48.942491 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e382d2-56e5-4e51-ae92-73c02936f92b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:48.943446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-669bb87f-8acd-4fd5-819d-8f08a1be69d8 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:48.944630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-669bb87f-8acd-4fd5-819d-8f08a1be69d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:48.952030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e382d2-56e5-4e51-ae92-73c02936f92b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:48.952538 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 787/1556] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:48.953643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57288b32-890f-4144-b393-97243ab6a5fa None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.953933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57288b32-890f-4144-b393-97243ab6a5fa None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.954210 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57288b32-890f-4144-b393-97243ab6a5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.956916 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4eb05-3b7b-4cc7-92e5-a4ea839bd4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:48.957559 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4eb05-3b7b-4cc7-92e5-a4ea839bd4be admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.957797 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4eb05-3b7b-4cc7-92e5-a4ea839bd4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.958077 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4eb05-3b7b-4cc7-92e5-a4ea839bd4be admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.959341 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd4eb05-3b7b-4cc7-92e5-a4ea839bd4be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:48.960986 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd4eb05-3b7b-4cc7-92e5-a4ea839bd4be admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:48.971087 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 770/1557] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:48.971771 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4eb05-3b7b-4cc7-92e5-a4ea839bd4be admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:48.975357 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-57288b32-890f-4144-b393-97243ab6a5fa None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:48.976552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-57288b32-890f-4144-b393-97243ab6a5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:48.984402 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4eb05-3b7b-4cc7-92e5-a4ea839bd4be admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:48.984882 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 788/1558] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:48.986153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d4663-64b4-44b1-992b-b57643c0be6f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.986445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d4663-64b4-44b1-992b-b57643c0be6f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.986726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d4663-64b4-44b1-992b-b57643c0be6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.992474 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e6868-f6c7-4f80-9b18-91248f583a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:48.993142 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e6868-f6c7-4f80-9b18-91248f583a19 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:48.993429 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e6868-f6c7-4f80-9b18-91248f583a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:48.993711 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e6868-f6c7-4f80-9b18-91248f583a19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:48.994436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e6868-f6c7-4f80-9b18-91248f583a19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:48.994748 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e6868-f6c7-4f80-9b18-91248f583a19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:48.996345 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e6868-f6c7-4f80-9b18-91248f583a19 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.004492 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-589d4663-64b4-44b1-992b-b57643c0be6f None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:49.006255 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-589d4663-64b4-44b1-992b-b57643c0be6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:49.009567 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 771/1559] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:49.010452 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e6868-f6c7-4f80-9b18-91248f583a19 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.014387 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 789/1560] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:49.015079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d4663-64b4-44b1-992b-b57643c0be6f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.020319 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e6868-f6c7-4f80-9b18-91248f583a19 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.025064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d4663-64b4-44b1-992b-b57643c0be6f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.025587 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4e700-21cc-4aaa-a816-01b4017065e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.026228 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4e700-21cc-4aaa-a816-01b4017065e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.026517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4e700-21cc-4aaa-a816-01b4017065e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.026794 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4e700-21cc-4aaa-a816-01b4017065e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.027477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb4e700-21cc-4aaa-a816-01b4017065e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.028491 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-eeb4e700-21cc-4aaa-a816-01b4017065e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:28:49.028491 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 18 10:28:49.028491 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: Jul 18 10:28:49.029670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b482a7c-25a7-45f5-bb81-cc540872ab03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.029804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb4e700-21cc-4aaa-a816-01b4017065e5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.030297 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b482a7c-25a7-45f5-bb81-cc540872ab03 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.030578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b482a7c-25a7-45f5-bb81-cc540872ab03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.030856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b482a7c-25a7-45f5-bb81-cc540872ab03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.031569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b482a7c-25a7-45f5-bb81-cc540872ab03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:49.031873 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b482a7c-25a7-45f5-bb81-cc540872ab03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.032913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-4b482a7c-25a7-45f5-bb81-cc540872ab03 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:28:49.032913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 18 10:28:49.032913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: Jul 18 10:28:49.033779 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b482a7c-25a7-45f5-bb81-cc540872ab03 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.048166 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 790/1561] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:49.049945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef977bfc-a207-4dbb-a36c-bab0f07cf708 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.050236 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef977bfc-a207-4dbb-a36c-bab0f07cf708 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.050641 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef977bfc-a207-4dbb-a36c-bab0f07cf708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.059048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 772/1562] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:28:49 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:49.063201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4e700-21cc-4aaa-a816-01b4017065e5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.075207 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ef977bfc-a207-4dbb-a36c-bab0f07cf708 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:49.076425 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ef977bfc-a207-4dbb-a36c-bab0f07cf708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:49.107803 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:49.129746 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4e700-21cc-4aaa-a816-01b4017065e5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.135017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f479a-4239-437a-b11e-3ca2f98329b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.135677 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f479a-4239-437a-b11e-3ca2f98329b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.135960 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f479a-4239-437a-b11e-3ca2f98329b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.136238 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f479a-4239-437a-b11e-3ca2f98329b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.137000 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f479a-4239-437a-b11e-3ca2f98329b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:49.137311 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f479a-4239-437a-b11e-3ca2f98329b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.137610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 791/1563] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:49.138450 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef977bfc-a207-4dbb-a36c-bab0f07cf708 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.138920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f479a-4239-437a-b11e-3ca2f98329b9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.144275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:49.148390 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef977bfc-a207-4dbb-a36c-bab0f07cf708 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.160050 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 773/1564] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:49.160050 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f479a-4239-437a-b11e-3ca2f98329b9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.161298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899c254-7a4a-4fd7-ae82-bc455b8f47a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.162105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899c254-7a4a-4fd7-ae82-bc455b8f47a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.162525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899c254-7a4a-4fd7-ae82-bc455b8f47a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.163023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899c254-7a4a-4fd7-ae82-bc455b8f47a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.163775 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899c254-7a4a-4fd7-ae82-bc455b8f47a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.165471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899c254-7a4a-4fd7-ae82-bc455b8f47a4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.170450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f479a-4239-437a-b11e-3ca2f98329b9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.175213 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f522b4-8935-478c-80b1-7bf157658d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.175887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 792/1565] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:49.176064 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f522b4-8935-478c-80b1-7bf157658d14 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.176450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f522b4-8935-478c-80b1-7bf157658d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.176874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f522b4-8935-478c-80b1-7bf157658d14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.177735 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f522b4-8935-478c-80b1-7bf157658d14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:49.178255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f522b4-8935-478c-80b1-7bf157658d14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.179712 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899c254-7a4a-4fd7-ae82-bc455b8f47a4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.180094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f522b4-8935-478c-80b1-7bf157658d14 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.189252 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899c254-7a4a-4fd7-ae82-bc455b8f47a4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.193436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 774/1566] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:49.194283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f522b4-8935-478c-80b1-7bf157658d14 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.194804 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadc473-a38c-4687-afee-226253fd1671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.195448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadc473-a38c-4687-afee-226253fd1671 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.195733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadc473-a38c-4687-afee-226253fd1671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.196012 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadc473-a38c-4687-afee-226253fd1671 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.196735 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeadc473-a38c-4687-afee-226253fd1671 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.198650 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeadc473-a38c-4687-afee-226253fd1671 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.210172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f522b4-8935-478c-80b1-7bf157658d14 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.214676 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4248a8-fc78-479a-a328-f45aa6e33281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.215438 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4248a8-fc78-479a-a328-f45aa6e33281 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.215761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4248a8-fc78-479a-a328-f45aa6e33281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.216129 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4248a8-fc78-479a-a328-f45aa6e33281 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.217029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4248a8-fc78-479a-a328-f45aa6e33281 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:49.217398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4248a8-fc78-479a-a328-f45aa6e33281 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.219142 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4248a8-fc78-479a-a328-f45aa6e33281 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.226039 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 793/1567] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:28:49 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:49.226872 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadc473-a38c-4687-afee-226253fd1671 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.239470 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 775/1568] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:49.239470 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4248a8-fc78-479a-a328-f45aa6e33281 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.244846 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4248a8-fc78-479a-a328-f45aa6e33281 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.289279 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadc473-a38c-4687-afee-226253fd1671 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.294669 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7deb85-dac0-474b-a25c-a261a902ad1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.295462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7deb85-dac0-474b-a25c-a261a902ad1f admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.295751 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7deb85-dac0-474b-a25c-a261a902ad1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.296048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7deb85-dac0-474b-a25c-a261a902ad1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.296772 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7deb85-dac0-474b-a25c-a261a902ad1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.298722 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7deb85-dac0-474b-a25c-a261a902ad1f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.300706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5200354-b720-43d1-a734-06f8ca32cb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.301315 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5200354-b720-43d1-a734-06f8ca32cb49 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.301639 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5200354-b720-43d1-a734-06f8ca32cb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.301949 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5200354-b720-43d1-a734-06f8ca32cb49 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.302683 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5200354-b720-43d1-a734-06f8ca32cb49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.304606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5200354-b720-43d1-a734-06f8ca32cb49 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.325713 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 794/1569] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:28:49 2022] POST /identity/v3/projects => generated 389 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:49.326743 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7deb85-dac0-474b-a25c-a261a902ad1f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.329059 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 776/1570] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:28:49 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:49.329900 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5200354-b720-43d1-a734-06f8ca32cb49 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.340422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5200354-b720-43d1-a734-06f8ca32cb49 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.390039 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7deb85-dac0-474b-a25c-a261a902ad1f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.394724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d2723-0681-410e-bee2-5409f3903a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.395486 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d2723-0681-410e-bee2-5409f3903a7b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.395701 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d2723-0681-410e-bee2-5409f3903a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.396008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d2723-0681-410e-bee2-5409f3903a7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.397048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d2723-0681-410e-bee2-5409f3903a7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:28:49.397378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d2723-0681-410e-bee2-5409f3903a7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.398114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcb8d3-9787-41e1-adec-23d2f1e2cfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.398857 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcb8d3-9787-41e1-adec-23d2f1e2cfa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.398965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d2723-0681-410e-bee2-5409f3903a7b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.399307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcb8d3-9787-41e1-adec-23d2f1e2cfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.399592 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcb8d3-9787-41e1-adec-23d2f1e2cfa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.400448 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bcb8d3-9787-41e1-adec-23d2f1e2cfa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.402837 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bcb8d3-9787-41e1-adec-23d2f1e2cfa1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.410349 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 795/1571] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:28:49.411125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d2723-0681-410e-bee2-5409f3903a7b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.420421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d2723-0681-410e-bee2-5409f3903a7b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.425135 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae0e1e-e79f-4d56-b552-e1b595266fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.426137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae0e1e-e79f-4d56-b552-e1b595266fcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.426448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae0e1e-e79f-4d56-b552-e1b595266fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.426760 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae0e1e-e79f-4d56-b552-e1b595266fcf admin admin] PATH_INFO: `/v3/projects/f2fc95f0534240e1a13971da652442ff/users/a6d07f10b21d483cb40ded0d83d74fa7/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.438818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 777/1572] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:28:49 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:49.438818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcb8d3-9787-41e1-adec-23d2f1e2cfa1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.442623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae0e1e-e79f-4d56-b552-e1b595266fcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2fc95f0534240e1a13971da652442ff, user_id=a6d07f10b21d483cb40ded0d83d74fa7, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.445972 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae0e1e-e79f-4d56-b552-e1b595266fcf admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.458240 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:49.458575 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 796/1573] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:28:49 2022] PUT /identity/v3/projects/f2fc95f0534240e1a13971da652442ff/users/a6d07f10b21d483cb40ded0d83d74fa7/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:49.459349 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae0e1e-e79f-4d56-b552-e1b595266fcf admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.488543 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:49.512586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcb8d3-9787-41e1-adec-23d2f1e2cfa1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.523991 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae0e1e-e79f-4d56-b552-e1b595266fcf admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.528738 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a814da0-d521-40a8-9263-bc1b8720e3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.529450 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a814da0-d521-40a8-9263-bc1b8720e3f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.529731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a814da0-d521-40a8-9263-bc1b8720e3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.530010 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a814da0-d521-40a8-9263-bc1b8720e3f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.530699 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a814da0-d521-40a8-9263-bc1b8720e3f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.531724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-1a814da0-d521-40a8-9263-bc1b8720e3f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:28:49.531724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 18 10:28:49.531724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: Jul 18 10:28:49.532961 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a814da0-d521-40a8-9263-bc1b8720e3f2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.551686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 797/1574] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:28:49 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:49.553033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a814da0-d521-40a8-9263-bc1b8720e3f2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.562386 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a814da0-d521-40a8-9263-bc1b8720e3f2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.567010 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69b03c-ac26-4ca0-864e-2fc28d0d45d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.567632 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69b03c-ac26-4ca0-864e-2fc28d0d45d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.567998 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69b03c-ac26-4ca0-864e-2fc28d0d45d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.568280 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69b03c-ac26-4ca0-864e-2fc28d0d45d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.569487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69b03c-ac26-4ca0-864e-2fc28d0d45d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.571377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69b03c-ac26-4ca0-864e-2fc28d0d45d5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.573162 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb676a-29ef-4a4a-851d-a113f6d3b006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.573831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb676a-29ef-4a4a-851d-a113f6d3b006 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.574117 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb676a-29ef-4a4a-851d-a113f6d3b006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.574575 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb676a-29ef-4a4a-851d-a113f6d3b006 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.575597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb676a-29ef-4a4a-851d-a113f6d3b006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.577533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb676a-29ef-4a4a-851d-a113f6d3b006 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.597878 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 798/1575] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:28:49 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:49.598853 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69b03c-ac26-4ca0-864e-2fc28d0d45d5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.604396 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 778/1576] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:28:49 2022] POST /identity/v3/projects => generated 363 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:49.605230 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb676a-29ef-4a4a-851d-a113f6d3b006 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.660797 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69b03c-ac26-4ca0-864e-2fc28d0d45d5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.665535 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312df985-f986-4c63-a59a-46c3bf9f2ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.666203 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312df985-f986-4c63-a59a-46c3bf9f2ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.666494 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312df985-f986-4c63-a59a-46c3bf9f2ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.666776 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312df985-f986-4c63-a59a-46c3bf9f2ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.666875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb676a-29ef-4a4a-851d-a113f6d3b006 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.667541 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312df985-f986-4c63-a59a-46c3bf9f2ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.669199 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312df985-f986-4c63-a59a-46c3bf9f2ce5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.671471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fa76d-7b03-48dc-bc71-73917c9b3918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.672160 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fa76d-7b03-48dc-bc71-73917c9b3918 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.672443 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fa76d-7b03-48dc-bc71-73917c9b3918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.672820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fa76d-7b03-48dc-bc71-73917c9b3918 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.673508 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fa76d-7b03-48dc-bc71-73917c9b3918 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.675410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fa76d-7b03-48dc-bc71-73917c9b3918 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.679639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 799/1577] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:49.680492 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312df985-f986-4c63-a59a-46c3bf9f2ce5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.696988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312df985-f986-4c63-a59a-46c3bf9f2ce5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.701055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 779/1578] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:28:49 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:49.701055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fa76d-7b03-48dc-bc71-73917c9b3918 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.701996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5faa854-6edd-4e7d-b221-937e0b7afc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.702654 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5faa854-6edd-4e7d-b221-937e0b7afc5f admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.702943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5faa854-6edd-4e7d-b221-937e0b7afc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.703277 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5faa854-6edd-4e7d-b221-937e0b7afc5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.703979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5faa854-6edd-4e7d-b221-937e0b7afc5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.705947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5faa854-6edd-4e7d-b221-937e0b7afc5f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.708328 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fa76d-7b03-48dc-bc71-73917c9b3918 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.712891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c8e6d-4e86-457d-b32c-46d41328a6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.713531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c8e6d-4e86-457d-b32c-46d41328a6d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.713813 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c8e6d-4e86-457d-b32c-46d41328a6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.714445 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c8e6d-4e86-457d-b32c-46d41328a6d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.714766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c8e6d-4e86-457d-b32c-46d41328a6d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.716908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c8e6d-4e86-457d-b32c-46d41328a6d0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.734527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 800/1579] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:28:49 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:49.735881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf92ca-9e0a-4c03-a6f4-69a479dc9a04 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.736178 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf92ca-9e0a-4c03-a6f4-69a479dc9a04 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.736459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf92ca-9e0a-4c03-a6f4-69a479dc9a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.739542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 780/1580] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:28:49 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:49.740924 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c8e6d-4e86-457d-b32c-46d41328a6d0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.750484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c8e6d-4e86-457d-b32c-46d41328a6d0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.765602 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d9bf92ca-9e0a-4c03-a6f4-69a479dc9a04 None None] MFA Rules not processed for user `a6d07f10b21d483cb40ded0d83d74fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:49.766872 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d9bf92ca-9e0a-4c03-a6f4-69a479dc9a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:49.771302 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:49.812187 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7044002-42cb-4768-9423-fa5af00aa88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.813195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7044002-42cb-4768-9423-fa5af00aa88b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.813488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7044002-42cb-4768-9423-fa5af00aa88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.813771 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7044002-42cb-4768-9423-fa5af00aa88b admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.814508 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7044002-42cb-4768-9423-fa5af00aa88b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.816125 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7044002-42cb-4768-9423-fa5af00aa88b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.826146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 781/1581] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:49.826934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7044002-42cb-4768-9423-fa5af00aa88b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.828196 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:49.834809 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 801/1582] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:49.835777 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf92ca-9e0a-4c03-a6f4-69a479dc9a04 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.840675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7044002-42cb-4768-9423-fa5af00aa88b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.846329 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf92ca-9e0a-4c03-a6f4-69a479dc9a04 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.846460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c71018-9bba-49b3-91fe-17095e400541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.847007 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c71018-9bba-49b3-91fe-17095e400541 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.847294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c71018-9bba-49b3-91fe-17095e400541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.847576 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c71018-9bba-49b3-91fe-17095e400541 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.848271 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c71018-9bba-49b3-91fe-17095e400541 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.850897 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c71018-9bba-49b3-91fe-17095e400541 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.851723 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83bd8c-cfdd-4525-8de1-8f9532b9cc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.852370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83bd8c-cfdd-4525-8de1-8f9532b9cc07 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.852686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83bd8c-cfdd-4525-8de1-8f9532b9cc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.852961 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83bd8c-cfdd-4525-8de1-8f9532b9cc07 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.853647 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83bd8c-cfdd-4525-8de1-8f9532b9cc07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.855791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83bd8c-cfdd-4525-8de1-8f9532b9cc07 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.872308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 782/1583] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:28:49 2022] POST /identity/v3/users => generated 289 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:49.876004 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 802/1584] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:28:49 2022] POST /identity/v3/users => generated 282 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:49.876143 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c71018-9bba-49b3-91fe-17095e400541 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.876972 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83bd8c-cfdd-4525-8de1-8f9532b9cc07 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.885707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c71018-9bba-49b3-91fe-17095e400541 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.888236 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83bd8c-cfdd-4525-8de1-8f9532b9cc07 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.890674 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545e094-76a7-4435-acbe-f1e433793b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.891474 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545e094-76a7-4435-acbe-f1e433793b2e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.891761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545e094-76a7-4435-acbe-f1e433793b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.892040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545e094-76a7-4435-acbe-f1e433793b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.892817 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545e094-76a7-4435-acbe-f1e433793b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.893581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8f80a-8c45-4132-8173-986ccbef4bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.894226 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8f80a-8c45-4132-8173-986ccbef4bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.894411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545e094-76a7-4435-acbe-f1e433793b2e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.894515 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8f80a-8c45-4132-8173-986ccbef4bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.894775 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8f80a-8c45-4132-8173-986ccbef4bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.895508 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8f80a-8c45-4132-8173-986ccbef4bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.897444 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8f80a-8c45-4132-8173-986ccbef4bc3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.905214 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 783/1585] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:49.905982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545e094-76a7-4435-acbe-f1e433793b2e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.907578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 803/1586] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:49.908491 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8f80a-8c45-4132-8173-986ccbef4bc3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.918630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8f80a-8c45-4132-8173-986ccbef4bc3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.918776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545e094-76a7-4435-acbe-f1e433793b2e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.923240 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34eebbf-e3d3-4503-842e-fd1d599ccd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.923527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29990fc1-d378-4398-b65a-1665a16a21bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.923883 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34eebbf-e3d3-4503-842e-fd1d599ccd26 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.924160 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29990fc1-d378-4398-b65a-1665a16a21bd admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.924259 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34eebbf-e3d3-4503-842e-fd1d599ccd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.924438 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34eebbf-e3d3-4503-842e-fd1d599ccd26 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.924565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29990fc1-d378-4398-b65a-1665a16a21bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.924749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29990fc1-d378-4398-b65a-1665a16a21bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.925168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34eebbf-e3d3-4503-842e-fd1d599ccd26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.925580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29990fc1-d378-4398-b65a-1665a16a21bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.927063 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34eebbf-e3d3-4503-842e-fd1d599ccd26 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.927826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29990fc1-d378-4398-b65a-1665a16a21bd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.938481 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 804/1587] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:49.941727 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29990fc1-d378-4398-b65a-1665a16a21bd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.949693 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29990fc1-d378-4398-b65a-1665a16a21bd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.955067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 784/1588] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:28:49 2022] POST /identity/v3/users => generated 292 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:49.955067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34eebbf-e3d3-4503-842e-fd1d599ccd26 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.955308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022b551-d7bf-44e7-b06a-d6d272c9e555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.955308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022b551-d7bf-44e7-b06a-d6d272c9e555 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.955308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022b551-d7bf-44e7-b06a-d6d272c9e555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.955550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022b551-d7bf-44e7-b06a-d6d272c9e555 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.956260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022b551-d7bf-44e7-b06a-d6d272c9e555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.957915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022b551-d7bf-44e7-b06a-d6d272c9e555 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.964237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34eebbf-e3d3-4503-842e-fd1d599ccd26 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.971716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 805/1589] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:49.971845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ae481-fad7-4839-b426-998b37754499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.971972 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022b551-d7bf-44e7-b06a-d6d272c9e555 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:49.972054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ae481-fad7-4839-b426-998b37754499 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.972054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ae481-fad7-4839-b426-998b37754499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.972054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ae481-fad7-4839-b426-998b37754499 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.972054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ae481-fad7-4839-b426-998b37754499 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.972922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ae481-fad7-4839-b426-998b37754499 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:49.982378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022b551-d7bf-44e7-b06a-d6d272c9e555 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:49.986921 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9730b7-e4af-49be-9d20-56dde2e90838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:49.987553 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9730b7-e4af-49be-9d20-56dde2e90838 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:49.987836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9730b7-e4af-49be-9d20-56dde2e90838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:49.988110 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9730b7-e4af-49be-9d20-56dde2e90838 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:49.988851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9730b7-e4af-49be-9d20-56dde2e90838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:49.991045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9730b7-e4af-49be-9d20-56dde2e90838 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.001506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 785/1590] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:28:49 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:50.002419 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ae481-fad7-4839-b426-998b37754499 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.003109 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 806/1591] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:50.003905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9730b7-e4af-49be-9d20-56dde2e90838 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.015189 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9730b7-e4af-49be-9d20-56dde2e90838 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.072583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ae481-fad7-4839-b426-998b37754499 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.072859 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83181ac-7f52-4b1e-b39a-ed9092adfc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.073516 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83181ac-7f52-4b1e-b39a-ed9092adfc69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.073831 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83181ac-7f52-4b1e-b39a-ed9092adfc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.074106 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83181ac-7f52-4b1e-b39a-ed9092adfc69 admin admin] PATH_INFO: `/v3/projects/23fa6d35c6354d7e9d12cf6bf8d33556/users/08f5f520abda4ad29f1841af7b6c8695/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.077302 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68624bb8-4eb0-4d51-b9ee-7a8de3e58e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.077973 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68624bb8-4eb0-4d51-b9ee-7a8de3e58e7b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.078261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68624bb8-4eb0-4d51-b9ee-7a8de3e58e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.078546 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68624bb8-4eb0-4d51-b9ee-7a8de3e58e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.079289 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68624bb8-4eb0-4d51-b9ee-7a8de3e58e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:50.080943 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68624bb8-4eb0-4d51-b9ee-7a8de3e58e7b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.089073 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83181ac-7f52-4b1e-b39a-ed9092adfc69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23fa6d35c6354d7e9d12cf6bf8d33556, user_id=08f5f520abda4ad29f1841af7b6c8695, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:50.091415 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 786/1592] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:50.092234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68624bb8-4eb0-4d51-b9ee-7a8de3e58e7b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.092773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83181ac-7f52-4b1e-b39a-ed9092adfc69 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.101857 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68624bb8-4eb0-4d51-b9ee-7a8de3e58e7b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.106760 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 807/1593] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:28:50 2022] PUT /identity/v3/projects/23fa6d35c6354d7e9d12cf6bf8d33556/users/08f5f520abda4ad29f1841af7b6c8695/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:50.109362 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83181ac-7f52-4b1e-b39a-ed9092adfc69 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.147735 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:50.161017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bf4f7-c2ad-45f1-982f-f48683c6db37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.161838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bf4f7-c2ad-45f1-982f-f48683c6db37 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.162120 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bf4f7-c2ad-45f1-982f-f48683c6db37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.162392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bf4f7-c2ad-45f1-982f-f48683c6db37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.163114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bf4f7-c2ad-45f1-982f-f48683c6db37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:50.164725 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bf4f7-c2ad-45f1-982f-f48683c6db37 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.174069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83181ac-7f52-4b1e-b39a-ed9092adfc69 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.174686 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 787/1594] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:50.175418 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bf4f7-c2ad-45f1-982f-f48683c6db37 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.183565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ca1be-40dd-40e0-919b-aa8abf08061e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.184256 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ca1be-40dd-40e0-919b-aa8abf08061e admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.184580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ca1be-40dd-40e0-919b-aa8abf08061e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.184861 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ca1be-40dd-40e0-919b-aa8abf08061e admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.185628 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ca1be-40dd-40e0-919b-aa8abf08061e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:50.185736 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bf4f7-c2ad-45f1-982f-f48683c6db37 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.187369 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ca1be-40dd-40e0-919b-aa8abf08061e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.190156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465a14f-ffc1-4d35-9024-9445bcd3ff18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.190804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465a14f-ffc1-4d35-9024-9445bcd3ff18 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.191085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465a14f-ffc1-4d35-9024-9445bcd3ff18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.191365 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465a14f-ffc1-4d35-9024-9445bcd3ff18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.192081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465a14f-ffc1-4d35-9024-9445bcd3ff18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:50.194244 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465a14f-ffc1-4d35-9024-9445bcd3ff18 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.198792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 808/1595] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:50.199695 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ca1be-40dd-40e0-919b-aa8abf08061e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.205703 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 788/1596] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:50.209277 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ca1be-40dd-40e0-919b-aa8abf08061e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.209437 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465a14f-ffc1-4d35-9024-9445bcd3ff18 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.214013 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570c990-9f76-4826-8371-a025a9b4633a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.214879 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570c990-9f76-4826-8371-a025a9b4633a admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.215172 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570c990-9f76-4826-8371-a025a9b4633a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.215456 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570c990-9f76-4826-8371-a025a9b4633a admin admin] PATH_INFO: `/v3/projects/926f2ef83f424b9f830078de3486361f/users/efcd904b529f44f1a7511d6d9009d40a/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.218967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465a14f-ffc1-4d35-9024-9445bcd3ff18 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.223590 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c26758-cf5a-4ef7-b009-f0f6d14369e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.224270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c26758-cf5a-4ef7-b009-f0f6d14369e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.224589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c26758-cf5a-4ef7-b009-f0f6d14369e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.224874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c26758-cf5a-4ef7-b009-f0f6d14369e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.225561 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c26758-cf5a-4ef7-b009-f0f6d14369e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:50.227529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c26758-cf5a-4ef7-b009-f0f6d14369e4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.227791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:50.231416 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570c990-9f76-4826-8371-a025a9b4633a admin admin] RBAC: Authorizing `identity:create_grant(project_id=926f2ef83f424b9f830078de3486361f, user_id=efcd904b529f44f1a7511d6d9009d40a, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:50.234561 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570c990-9f76-4826-8371-a025a9b4633a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.249289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 809/1597] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:28:50 2022] PUT /identity/v3/projects/926f2ef83f424b9f830078de3486361f/users/efcd904b529f44f1a7511d6d9009d40a/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:50.249784 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 789/1598] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:28:50 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:50.249943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570c990-9f76-4826-8371-a025a9b4633a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.250532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c26758-cf5a-4ef7-b009-f0f6d14369e4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.265986 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c26758-cf5a-4ef7-b009-f0f6d14369e4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.313731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570c990-9f76-4826-8371-a025a9b4633a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.319004 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc047c-95a4-4147-b8cd-9e8b1d158d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.319681 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc047c-95a4-4147-b8cd-9e8b1d158d42 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.319962 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc047c-95a4-4147-b8cd-9e8b1d158d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.320244 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc047c-95a4-4147-b8cd-9e8b1d158d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.321001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc047c-95a4-4147-b8cd-9e8b1d158d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:50.322609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc047c-95a4-4147-b8cd-9e8b1d158d42 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.326094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cac205-e140-43ab-9f93-c369bc2e0241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.326761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cac205-e140-43ab-9f93-c369bc2e0241 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.327047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cac205-e140-43ab-9f93-c369bc2e0241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.327326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cac205-e140-43ab-9f93-c369bc2e0241 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.328081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cac205-e140-43ab-9f93-c369bc2e0241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:50.329699 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cac205-e140-43ab-9f93-c369bc2e0241 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.332474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 810/1599] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:50.336989 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faca6b8-9a82-4931-9d94-08a6008abdb2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.337202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faca6b8-9a82-4931-9d94-08a6008abdb2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.337491 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faca6b8-9a82-4931-9d94-08a6008abdb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.342507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 790/1600] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:50.342507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cac205-e140-43ab-9f93-c369bc2e0241 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.352758 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cac205-e140-43ab-9f93-c369bc2e0241 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.357709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6b716-07d9-4cfa-941c-138f85c7fe90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.358593 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6b716-07d9-4cfa-941c-138f85c7fe90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.359579 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6b716-07d9-4cfa-941c-138f85c7fe90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.359579 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6b716-07d9-4cfa-941c-138f85c7fe90 admin admin] PATH_INFO: `/v3/projects/8a1921237622439bab89676e11477c13/users/72a431c9708f4694bc0d514133e76088/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.366865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8faca6b8-9a82-4931-9d94-08a6008abdb2 None None] MFA Rules not processed for user `08f5f520abda4ad29f1841af7b6c8695`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:50.368078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8faca6b8-9a82-4931-9d94-08a6008abdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:50.374359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6b716-07d9-4cfa-941c-138f85c7fe90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a1921237622439bab89676e11477c13, user_id=72a431c9708f4694bc0d514133e76088, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:50.375432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-0cf6b716-07d9-4cfa-941c-138f85c7fe90 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:28:50.375432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 18 10:28:50.375432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: Jul 18 10:28:50.377491 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6b716-07d9-4cfa-941c-138f85c7fe90 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.391698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 791/1601] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:28:50 2022] PUT /identity/v3/projects/8a1921237622439bab89676e11477c13/users/72a431c9708f4694bc0d514133e76088/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:50.392687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6b716-07d9-4cfa-941c-138f85c7fe90 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.434074 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 811/1602] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:50.434074 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faca6b8-9a82-4931-9d94-08a6008abdb2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.445707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faca6b8-9a82-4931-9d94-08a6008abdb2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.450333 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54ced5-6ef7-428b-847b-4e7ddd09f8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.450979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54ced5-6ef7-428b-847b-4e7ddd09f8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.451268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54ced5-6ef7-428b-847b-4e7ddd09f8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.451550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54ced5-6ef7-428b-847b-4e7ddd09f8c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.452272 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae54ced5-6ef7-428b-847b-4e7ddd09f8c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:50.453722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6b716-07d9-4cfa-941c-138f85c7fe90 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.453903 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae54ced5-6ef7-428b-847b-4e7ddd09f8c4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.458252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885b4e2-e403-4cad-ae21-7529e79ba7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.459256 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885b4e2-e403-4cad-ae21-7529e79ba7a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.459515 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885b4e2-e403-4cad-ae21-7529e79ba7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.459795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885b4e2-e403-4cad-ae21-7529e79ba7a0 admin admin] PATH_INFO: `/v3/projects/daab623e57e54081b262d4a8fe124f42/users/c0f993e44c624f65abae22540ce7dffc/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.463776 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 812/1603] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:50.464600 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54ced5-6ef7-428b-847b-4e7ddd09f8c4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.474280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885b4e2-e403-4cad-ae21-7529e79ba7a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daab623e57e54081b262d4a8fe124f42, user_id=c0f993e44c624f65abae22540ce7dffc, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:50.475669 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54ced5-6ef7-428b-847b-4e7ddd09f8c4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.476983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885b4e2-e403-4cad-ae21-7529e79ba7a0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.481234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e4dd1-d8c9-46bf-ba86-34fa9d25de11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.481736 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e4dd1-d8c9-46bf-ba86-34fa9d25de11 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.482106 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e4dd1-d8c9-46bf-ba86-34fa9d25de11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.482296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e4dd1-d8c9-46bf-ba86-34fa9d25de11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.483088 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e4dd1-d8c9-46bf-ba86-34fa9d25de11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:50.484832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e4dd1-d8c9-46bf-ba86-34fa9d25de11 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.491592 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 792/1604] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:28:50 2022] PUT /identity/v3/projects/daab623e57e54081b262d4a8fe124f42/users/c0f993e44c624f65abae22540ce7dffc/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:50.492868 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c33454-0c28-4901-9e0d-7affaece9fe2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.493161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c33454-0c28-4901-9e0d-7affaece9fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.493440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c33454-0c28-4901-9e0d-7affaece9fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.499062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 813/1605] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:50.499062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e4dd1-d8c9-46bf-ba86-34fa9d25de11 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.511045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e4dd1-d8c9-46bf-ba86-34fa9d25de11 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.516246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:50.525567 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-90c33454-0c28-4901-9e0d-7affaece9fe2 None None] MFA Rules not processed for user `efcd904b529f44f1a7511d6d9009d40a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:50.527500 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-90c33454-0c28-4901-9e0d-7affaece9fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:50.566995 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:50.575408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff47982-4dbc-45cf-b4cd-a17ca571d048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.576306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff47982-4dbc-45cf-b4cd-a17ca571d048 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.576626 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff47982-4dbc-45cf-b4cd-a17ca571d048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.576910 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff47982-4dbc-45cf-b4cd-a17ca571d048 admin admin] PATH_INFO: `/v3/projects/e6e487deb7ff45bdbf2cc002d42d12a5/users/986f90dfbad14324b3fffbd790d60a37/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.591327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff47982-4dbc-45cf-b4cd-a17ca571d048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6e487deb7ff45bdbf2cc002d42d12a5, user_id=986f90dfbad14324b3fffbd790d60a37, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:50.594162 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff47982-4dbc-45cf-b4cd-a17ca571d048 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.594811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 793/1606] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:50.595733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c33454-0c28-4901-9e0d-7affaece9fe2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.607263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c33454-0c28-4901-9e0d-7affaece9fe2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.611506 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 814/1607] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:28:50 2022] PUT /identity/v3/projects/e6e487deb7ff45bdbf2cc002d42d12a5/users/986f90dfbad14324b3fffbd790d60a37/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:50.618123 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a58f8-5246-4c59-91d3-af123dbeea39 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.618425 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a58f8-5246-4c59-91d3-af123dbeea39 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.618709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a58f8-5246-4c59-91d3-af123dbeea39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.638825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a47a58f8-5246-4c59-91d3-af123dbeea39 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:50.640049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a47a58f8-5246-4c59-91d3-af123dbeea39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:50.670361 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12f5e8-f967-43f5-9f2d-f40801f48ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.671282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12f5e8-f967-43f5-9f2d-f40801f48ede admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.671572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12f5e8-f967-43f5-9f2d-f40801f48ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.671852 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12f5e8-f967-43f5-9f2d-f40801f48ede admin admin] PATH_INFO: `/v3/projects/ec5436d4818a4748894ba4c46ec362d9/users/740613cc32554016ad9ae8f00849aa88/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.686781 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12f5e8-f967-43f5-9f2d-f40801f48ede admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec5436d4818a4748894ba4c46ec362d9, user_id=740613cc32554016ad9ae8f00849aa88, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:50.689522 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12f5e8-f967-43f5-9f2d-f40801f48ede admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.699060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 815/1608] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:50.700122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a58f8-5246-4c59-91d3-af123dbeea39 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.705839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 794/1609] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:28:50 2022] PUT /identity/v3/projects/ec5436d4818a4748894ba4c46ec362d9/users/740613cc32554016ad9ae8f00849aa88/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:50.707208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12f5e8-f967-43f5-9f2d-f40801f48ede admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.713055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a58f8-5246-4c59-91d3-af123dbeea39 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.771648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d237d0c-ac68-4ccf-a7e1-c31d2f2f3531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.772221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d237d0c-ac68-4ccf-a7e1-c31d2f2f3531 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.772524 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d237d0c-ac68-4ccf-a7e1-c31d2f2f3531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.772810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d237d0c-ac68-4ccf-a7e1-c31d2f2f3531 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.773523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d237d0c-ac68-4ccf-a7e1-c31d2f2f3531 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:50.774696 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12f5e8-f967-43f5-9f2d-f40801f48ede admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.775435 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d237d0c-ac68-4ccf-a7e1-c31d2f2f3531 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.780028 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada477f-d460-4fb5-a071-c4a6314fe57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.780953 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada477f-d460-4fb5-a071-c4a6314fe57d admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.781246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada477f-d460-4fb5-a071-c4a6314fe57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.781528 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada477f-d460-4fb5-a071-c4a6314fe57d admin admin] PATH_INFO: `/v3/projects/1cf3d5bfe9a745f38372c42c9cfa1f63/users/f25c72896a2d4d619e5ad4b5a0a79aa6/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.797364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada477f-d460-4fb5-a071-c4a6314fe57d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf3d5bfe9a745f38372c42c9cfa1f63, user_id=f25c72896a2d4d619e5ad4b5a0a79aa6, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:50.800044 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada477f-d460-4fb5-a071-c4a6314fe57d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.804297 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 816/1610] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:28:50 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:50.804297 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3ebc4-d711-4be8-ad3b-d5a9f1f4e4a5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.804297 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3ebc4-d711-4be8-ad3b-d5a9f1f4e4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.804297 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3ebc4-d711-4be8-ad3b-d5a9f1f4e4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.815845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 795/1611] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:28:50 2022] PUT /identity/v3/projects/1cf3d5bfe9a745f38372c42c9cfa1f63/users/f25c72896a2d4d619e5ad4b5a0a79aa6/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:50.816427 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada477f-d460-4fb5-a071-c4a6314fe57d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.821439 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-08a3ebc4-d711-4be8-ad3b-d5a9f1f4e4a5 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:50.822654 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-08a3ebc4-d711-4be8-ad3b-d5a9f1f4e4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:50.824862 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:50.878922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada477f-d460-4fb5-a071-c4a6314fe57d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.883569 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954e691-4f31-465c-a8e7-c649cc5448d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.884469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954e691-4f31-465c-a8e7-c649cc5448d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.884883 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 817/1612] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:50.885025 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954e691-4f31-465c-a8e7-c649cc5448d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.885025 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954e691-4f31-465c-a8e7-c649cc5448d4 admin admin] PATH_INFO: `/v3/projects/f18232497e5c4f37beb904eca31303f8/users/cb1b85f84b324a7eaf8266c9ef8108a8/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.886022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7e98e-f7e1-4980-871d-1f5fd89a9ca8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.886396 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7e98e-f7e1-4980-871d-1f5fd89a9ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.886678 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7e98e-f7e1-4980-871d-1f5fd89a9ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.899391 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:50.901045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954e691-4f31-465c-a8e7-c649cc5448d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f18232497e5c4f37beb904eca31303f8, user_id=cb1b85f84b324a7eaf8266c9ef8108a8, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:50.904371 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954e691-4f31-465c-a8e7-c649cc5448d4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:50.912233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-33d7e98e-f7e1-4980-871d-1f5fd89a9ca8 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:50.913694 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-33d7e98e-f7e1-4980-871d-1f5fd89a9ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:50.917676 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 796/1613] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:28:50 2022] PUT /identity/v3/projects/f18232497e5c4f37beb904eca31303f8/users/cb1b85f84b324a7eaf8266c9ef8108a8/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:50.921796 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f9446-bdf8-46a1-bd55-50e9bd357863 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.922094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f9446-bdf8-46a1-bd55-50e9bd357863 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.922378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f9446-bdf8-46a1-bd55-50e9bd357863 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.923658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 797/1614] 10.222.0.33 () {54 vars in 962 bytes} [Mon Jul 18 10:28:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:28:50.925381 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f9446-bdf8-46a1-bd55-50e9bd357863 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:50.974674 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 818/1615] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:50.976343 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68088118-9939-434f-93bb-983951bbc560 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.976741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68088118-9939-434f-93bb-983951bbc560 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.977023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68088118-9939-434f-93bb-983951bbc560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:50.989768 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f9446-bdf8-46a1-bd55-50e9bd357863 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:50.993554 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ff247-0fd1-475a-b12a-969d95efc8a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:50.994367 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ff247-0fd1-475a-b12a-969d95efc8a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:50.994650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ff247-0fd1-475a-b12a-969d95efc8a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:50.994926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ff247-0fd1-475a-b12a-969d95efc8a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.004151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-68088118-9939-434f-93bb-983951bbc560 None None] MFA Rules not processed for user `740613cc32554016ad9ae8f00849aa88`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:51.005441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ff247-0fd1-475a-b12a-969d95efc8a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.005583 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-68088118-9939-434f-93bb-983951bbc560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:51.007044 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ff247-0fd1-475a-b12a-969d95efc8a4 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.017506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 798/1616] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:28:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:28:51.018289 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ff247-0fd1-475a-b12a-969d95efc8a4 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.029428 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ff247-0fd1-475a-b12a-969d95efc8a4 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.034317 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f6326-6039-48aa-b097-54aa50f637cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.034987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f6326-6039-48aa-b097-54aa50f637cf admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.035271 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f6326-6039-48aa-b097-54aa50f637cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.035551 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f6326-6039-48aa-b097-54aa50f637cf admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.036243 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f6326-6039-48aa-b097-54aa50f637cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.038776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f6326-6039-48aa-b097-54aa50f637cf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.061272 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 799/1617] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:28:51 2022] POST /identity/v3/users => generated 295 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:51.062533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc57fd4-7a58-40ec-8d92-0ddf01eb9fa7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.062965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc57fd4-7a58-40ec-8d92-0ddf01eb9fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.063260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc57fd4-7a58-40ec-8d92-0ddf01eb9fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.081149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6bc57fd4-7a58-40ec-8d92-0ddf01eb9fa7 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:51.082379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6bc57fd4-7a58-40ec-8d92-0ddf01eb9fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:51.085631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 819/1618] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:50 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:51.086388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68088118-9939-434f-93bb-983951bbc560 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.096844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68088118-9939-434f-93bb-983951bbc560 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.099815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 800/1619] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:51.100920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc57fd4-7a58-40ec-8d92-0ddf01eb9fa7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.110204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079fe5c-2593-4744-91b8-39d97db35608 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.110870 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079fe5c-2593-4744-91b8-39d97db35608 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.111181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079fe5c-2593-4744-91b8-39d97db35608 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.111285 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc57fd4-7a58-40ec-8d92-0ddf01eb9fa7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.111484 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079fe5c-2593-4744-91b8-39d97db35608 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.116270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1e18b-49c5-4cfb-bf46-7ac0c5c7fee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.116984 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1e18b-49c5-4cfb-bf46-7ac0c5c7fee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.117294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1e18b-49c5-4cfb-bf46-7ac0c5c7fee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.117552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1e18b-49c5-4cfb-bf46-7ac0c5c7fee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.118355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1e18b-49c5-4cfb-bf46-7ac0c5c7fee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.120011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1e18b-49c5-4cfb-bf46-7ac0c5c7fee2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.122513 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079fe5c-2593-4744-91b8-39d97db35608 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.123965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079fe5c-2593-4744-91b8-39d97db35608 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.130719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 801/1620] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:51.131937 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d70c4-5cd3-443d-92d6-d1b819710e3f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.132238 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d70c4-5cd3-443d-92d6-d1b819710e3f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.132556 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d70c4-5cd3-443d-92d6-d1b819710e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.140838 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 820/1621] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:28:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:28:51.142614 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079fe5c-2593-4744-91b8-39d97db35608 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.156857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079fe5c-2593-4744-91b8-39d97db35608 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.161471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be924ba9-e8ef-4083-9259-219ed40ef649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.162658 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be924ba9-e8ef-4083-9259-219ed40ef649 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.162989 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be924ba9-e8ef-4083-9259-219ed40ef649 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.163267 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be924ba9-e8ef-4083-9259-219ed40ef649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.176842 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be924ba9-e8ef-4083-9259-219ed40ef649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.177174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-614d70c4-5cd3-443d-92d6-d1b819710e3f None None] MFA Rules not processed for user `29048c2021e340b081ae44a390583e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:51.178450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-614d70c4-5cd3-443d-92d6-d1b819710e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:51.179379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be924ba9-e8ef-4083-9259-219ed40ef649 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.194989 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 821/1622] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:28:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:28:51.196193 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be924ba9-e8ef-4083-9259-219ed40ef649 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.209890 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be924ba9-e8ef-4083-9259-219ed40ef649 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.215144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a45c31-9766-43da-a27b-cd72aa658bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.216074 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a45c31-9766-43da-a27b-cd72aa658bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.216361 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a45c31-9766-43da-a27b-cd72aa658bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.216671 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a45c31-9766-43da-a27b-cd72aa658bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.217471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a45c31-9766-43da-a27b-cd72aa658bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.219044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a45c31-9766-43da-a27b-cd72aa658bb3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.230191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:51.232007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 822/1623] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:51.232007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a45c31-9766-43da-a27b-cd72aa658bb3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.242576 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a45c31-9766-43da-a27b-cd72aa658bb3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.248423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7d2d9-3679-4709-991f-7d03f65deb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.249111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7d2d9-3679-4709-991f-7d03f65deb97 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.249558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7d2d9-3679-4709-991f-7d03f65deb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.249694 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7d2d9-3679-4709-991f-7d03f65deb97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.250337 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7d2d9-3679-4709-991f-7d03f65deb97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.252900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7d2d9-3679-4709-991f-7d03f65deb97 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.278295 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 802/1624] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Jul 18 10:28:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:51.281729 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d70c4-5cd3-443d-92d6-d1b819710e3f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.283345 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-614d70c4-5cd3-443d-92d6-d1b819710e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:51.286361 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 823/1625] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:28:51 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:51.287278 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7d2d9-3679-4709-991f-7d03f65deb97 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.299662 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d70c4-5cd3-443d-92d6-d1b819710e3f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.329870 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:51.362580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7d2d9-3679-4709-991f-7d03f65deb97 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.368530 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086f471-007e-4bfe-93f4-a6f5eaac20b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.369895 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086f471-007e-4bfe-93f4-a6f5eaac20b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.370543 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a713cde-1a77-4335-aeaf-02fe0eab1fd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.371014 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086f471-007e-4bfe-93f4-a6f5eaac20b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.371112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a713cde-1a77-4335-aeaf-02fe0eab1fd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.371191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086f471-007e-4bfe-93f4-a6f5eaac20b1 admin admin] PATH_INFO: `/v3/projects/f18232497e5c4f37beb904eca31303f8/users/cb1b85f84b324a7eaf8266c9ef8108a8/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.371267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a713cde-1a77-4335-aeaf-02fe0eab1fd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.371476 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a713cde-1a77-4335-aeaf-02fe0eab1fd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.373551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1086f471-007e-4bfe-93f4-a6f5eaac20b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f18232497e5c4f37beb904eca31303f8, user_id=cb1b85f84b324a7eaf8266c9ef8108a8, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.376372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1086f471-007e-4bfe-93f4-a6f5eaac20b1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.383542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a713cde-1a77-4335-aeaf-02fe0eab1fd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.385822 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a713cde-1a77-4335-aeaf-02fe0eab1fd6 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.391194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 824/1626] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:28:51 2022] PUT /identity/v3/projects/f18232497e5c4f37beb904eca31303f8/users/cb1b85f84b324a7eaf8266c9ef8108a8/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:51.392692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086f471-007e-4bfe-93f4-a6f5eaac20b1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.406205 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 803/1627] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:28:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:28:51.416685 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a713cde-1a77-4335-aeaf-02fe0eab1fd6 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.487927 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086f471-007e-4bfe-93f4-a6f5eaac20b1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.495577 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b995d0-6ce1-459f-a034-8209434b32f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.496902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b995d0-6ce1-459f-a034-8209434b32f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.497272 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b995d0-6ce1-459f-a034-8209434b32f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.498245 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b995d0-6ce1-459f-a034-8209434b32f8 admin admin] PATH_INFO: `/v3/projects/7d7b922345004824ba28ccfdbd80b5c0/users/eb9f9fd743b046f387b5b7a832b917ea/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.522793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b995d0-6ce1-459f-a034-8209434b32f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d7b922345004824ba28ccfdbd80b5c0, user_id=eb9f9fd743b046f387b5b7a832b917ea, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.523428 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a713cde-1a77-4335-aeaf-02fe0eab1fd6 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.527126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b995d0-6ce1-459f-a034-8209434b32f8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.549794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 825/1628] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:28:51 2022] PUT /identity/v3/projects/7d7b922345004824ba28ccfdbd80b5c0/users/eb9f9fd743b046f387b5b7a832b917ea/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:51.550445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b995d0-6ce1-459f-a034-8209434b32f8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.642340 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b995d0-6ce1-459f-a034-8209434b32f8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.647753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83433c-bed4-407d-b89d-1b377b856800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.648455 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83433c-bed4-407d-b89d-1b377b856800 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.648790 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83433c-bed4-407d-b89d-1b377b856800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.649076 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83433c-bed4-407d-b89d-1b377b856800 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.649781 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f83433c-bed4-407d-b89d-1b377b856800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.651817 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f83433c-bed4-407d-b89d-1b377b856800 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.658268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfd7a4-789e-48bc-a687-600e6bece8f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.660651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfd7a4-789e-48bc-a687-600e6bece8f8 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.661929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfd7a4-789e-48bc-a687-600e6bece8f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.663399 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfd7a4-789e-48bc-a687-600e6bece8f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.689361 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:51.700041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 826/1629] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:28:51 2022] POST /identity/v3/users => generated 288 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:28:51.704210 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f34384-aa2c-426c-a57c-a0771cdb2fbb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.704493 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f34384-aa2c-426c-a57c-a0771cdb2fbb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.705311 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f34384-aa2c-426c-a57c-a0771cdb2fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.754273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a6f34384-aa2c-426c-a57c-a0771cdb2fbb None None] MFA Rules not processed for user `cb1b85f84b324a7eaf8266c9ef8108a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:51.757139 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a6f34384-aa2c-426c-a57c-a0771cdb2fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:51.811390 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:51.833625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdfd7a4-789e-48bc-a687-600e6bece8f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.835761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdfd7a4-789e-48bc-a687-600e6bece8f8 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.867029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 804/1630] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:28:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:28:51.871388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfd7a4-789e-48bc-a687-600e6bece8f8 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.885366 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 827/1631] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:51.886451 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f34384-aa2c-426c-a57c-a0771cdb2fbb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.901655 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfd7a4-789e-48bc-a687-600e6bece8f8 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.906198 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f34384-aa2c-426c-a57c-a0771cdb2fbb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.906931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84d4db-43ca-47fd-85bf-f62fac09cbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.907802 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84d4db-43ca-47fd-85bf-f62fac09cbd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.907929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84d4db-43ca-47fd-85bf-f62fac09cbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.908144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84d4db-43ca-47fd-85bf-f62fac09cbd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.908969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f84d4db-43ca-47fd-85bf-f62fac09cbd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.910874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f84d4db-43ca-47fd-85bf-f62fac09cbd9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.912219 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc28e364-a41d-4cb4-9518-83d66887778c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.913437 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28e364-a41d-4cb4-9518-83d66887778c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.914230 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28e364-a41d-4cb4-9518-83d66887778c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.914662 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28e364-a41d-4cb4-9518-83d66887778c admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.915610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc28e364-a41d-4cb4-9518-83d66887778c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.918506 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc28e364-a41d-4cb4-9518-83d66887778c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.924017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 805/1632] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:51.924913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84d4db-43ca-47fd-85bf-f62fac09cbd9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.934406 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 828/1633] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:51.935326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc28e364-a41d-4cb4-9518-83d66887778c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:51.951756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc28e364-a41d-4cb4-9518-83d66887778c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:51.959635 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58464fbe-4fb1-46ab-98d4-b5a8a7f2caed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:51.963109 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58464fbe-4fb1-46ab-98d4-b5a8a7f2caed admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:51.963417 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58464fbe-4fb1-46ab-98d4-b5a8a7f2caed admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:51.963677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58464fbe-4fb1-46ab-98d4-b5a8a7f2caed admin admin] PATH_INFO: `/v3/projects/7d7b922345004824ba28ccfdbd80b5c0/users/eb9f9fd743b046f387b5b7a832b917ea/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:51.969586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58464fbe-4fb1-46ab-98d4-b5a8a7f2caed admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d7b922345004824ba28ccfdbd80b5c0, user_id=eb9f9fd743b046f387b5b7a832b917ea, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:51.972451 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58464fbe-4fb1-46ab-98d4-b5a8a7f2caed admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:51.986652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 829/1634] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:28:51 2022] PUT /identity/v3/projects/7d7b922345004824ba28ccfdbd80b5c0/users/eb9f9fd743b046f387b5b7a832b917ea/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:51.987598 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58464fbe-4fb1-46ab-98d4-b5a8a7f2caed admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:52.016934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84d4db-43ca-47fd-85bf-f62fac09cbd9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:52.074639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58464fbe-4fb1-46ab-98d4-b5a8a7f2caed admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:52.081925 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f4497-c782-49f2-ad34-afab94240ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:52.082897 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f4497-c782-49f2-ad34-afab94240ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:52.083185 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f4497-c782-49f2-ad34-afab94240ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:52.083499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f4497-c782-49f2-ad34-afab94240ec5 admin admin] PATH_INFO: `/v3/projects/c1b45fbfe4704b8a8a0415be67da69b5/users/335bc08f3c544d0596a2ba7c8b77247a/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:52.085115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f323fd-7879-4c93-8fff-b1619087ad17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:52.085956 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f323fd-7879-4c93-8fff-b1619087ad17 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:52.086252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f323fd-7879-4c93-8fff-b1619087ad17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:52.086661 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f323fd-7879-4c93-8fff-b1619087ad17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:52.118346 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f4497-c782-49f2-ad34-afab94240ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b45fbfe4704b8a8a0415be67da69b5, user_id=335bc08f3c544d0596a2ba7c8b77247a, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:52.122960 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f4497-c782-49f2-ad34-afab94240ec5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:52.159363 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 830/1635] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:28:51 2022] PUT /identity/v3/projects/c1b45fbfe4704b8a8a0415be67da69b5/users/335bc08f3c544d0596a2ba7c8b77247a/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:52.163386 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cacaa-e45c-48da-90ec-f131a8922c11 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:52.163693 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cacaa-e45c-48da-90ec-f131a8922c11 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:52.163977 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cacaa-e45c-48da-90ec-f131a8922c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:52.192020 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f323fd-7879-4c93-8fff-b1619087ad17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:52.193738 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f323fd-7879-4c93-8fff-b1619087ad17 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:52.207576 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 806/1636] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:28:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:28:52.209505 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f323fd-7879-4c93-8fff-b1619087ad17 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:52.222714 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f323fd-7879-4c93-8fff-b1619087ad17 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:52.229266 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:52.245608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-730cacaa-e45c-48da-90ec-f131a8922c11 None None] MFA Rules not processed for user `eb9f9fd743b046f387b5b7a832b917ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:52.246165 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-730cacaa-e45c-48da-90ec-f131a8922c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:52.307708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f25d7-15a8-415a-b8b3-ac364741f9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:52.308452 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f25d7-15a8-415a-b8b3-ac364741f9be admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:52.308783 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f25d7-15a8-415a-b8b3-ac364741f9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:52.309069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f25d7-15a8-415a-b8b3-ac364741f9be admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:52.309815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f25d7-15a8-415a-b8b3-ac364741f9be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:52.311725 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f25d7-15a8-415a-b8b3-ac364741f9be admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:52.326066 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 807/1637] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:28:52.331618 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fd630-f237-4d87-9e24-9e06493d3c65 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:52.332336 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fd630-f237-4d87-9e24-9e06493d3c65 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:52.332658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fd630-f237-4d87-9e24-9e06493d3c65 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:52.334910 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 808/1638] 10.222.0.33 () {54 vars in 912 bytes} [Mon Jul 18 10:28:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:28:52.335679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fd630-f237-4d87-9e24-9e06493d3c65 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:52.352446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fd630-f237-4d87-9e24-9e06493d3c65 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:52.360027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ecec0-2b6e-476c-803d-6b21460fdd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:52.361593 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ecec0-2b6e-476c-803d-6b21460fdd68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:52.361593 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ecec0-2b6e-476c-803d-6b21460fdd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:52.361974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ecec0-2b6e-476c-803d-6b21460fdd68 admin admin] PATH_INFO: `/v3/projects/c1b45fbfe4704b8a8a0415be67da69b5/users/335bc08f3c544d0596a2ba7c8b77247a/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:52.366079 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ecec0-2b6e-476c-803d-6b21460fdd68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b45fbfe4704b8a8a0415be67da69b5, user_id=335bc08f3c544d0596a2ba7c8b77247a, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:52.369267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ecec0-2b6e-476c-803d-6b21460fdd68 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:52.394211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 809/1639] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:28:52 2022] PUT /identity/v3/projects/c1b45fbfe4704b8a8a0415be67da69b5/users/335bc08f3c544d0596a2ba7c8b77247a/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:52.396725 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfea58-cc57-434f-9ceb-97511b4394de None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:52.397080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfea58-cc57-434f-9ceb-97511b4394de None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:52.397488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfea58-cc57-434f-9ceb-97511b4394de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:52.397858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 831/1640] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:52 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:52.399517 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13808272-71ee-42d3-bda9-35ef456f997a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:52.399759 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13808272-71ee-42d3-bda9-35ef456f997a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:52.400036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13808272-71ee-42d3-bda9-35ef456f997a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:52.423390 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b6dfea58-cc57-434f-9ceb-97511b4394de None None] MFA Rules not processed for user `29048c2021e340b081ae44a390583e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:52.426098 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b6dfea58-cc57-434f-9ceb-97511b4394de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:52.455852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:52.469016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-13808272-71ee-42d3-bda9-35ef456f997a None None] MFA Rules not processed for user `335bc08f3c544d0596a2ba7c8b77247a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:52.470812 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-13808272-71ee-42d3-bda9-35ef456f997a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:52.586607 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 810/1641] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 18 10:28:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:52.586607 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dfea58-cc57-434f-9ceb-97511b4394de None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:52.648400 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 832/1642] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:52 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:52.649367 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13808272-71ee-42d3-bda9-35ef456f997a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:52.685953 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dfea58-cc57-434f-9ceb-97511b4394de None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:52.691791 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9ae0f-957a-4f13-a714-084ea821f632 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:52.693259 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9ae0f-957a-4f13-a714-084ea821f632 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:52.693582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9ae0f-957a-4f13-a714-084ea821f632 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:52.693828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9ae0f-957a-4f13-a714-084ea821f632 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:52.727328 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13808272-71ee-42d3-bda9-35ef456f997a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:52.732440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c63b70-e6b2-4cdb-a1ef-2f0bdb09801f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:52.733569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c63b70-e6b2-4cdb-a1ef-2f0bdb09801f service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:52.734190 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c63b70-e6b2-4cdb-a1ef-2f0bdb09801f service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:52.734674 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c63b70-e6b2-4cdb-a1ef-2f0bdb09801f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:52.749968 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c63b70-e6b2-4cdb-a1ef-2f0bdb09801f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:52.752074 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c63b70-e6b2-4cdb-a1ef-2f0bdb09801f service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:52.753400 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:52.773272 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 833/1643] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:28:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:28:52.795324 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9ae0f-957a-4f13-a714-084ea821f632 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:52.798248 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9ae0f-957a-4f13-a714-084ea821f632 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:52.816672 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 811/1644] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:28:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:28:53.671067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c63b70-e6b2-4cdb-a1ef-2f0bdb09801f service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:53.683987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c63b70-e6b2-4cdb-a1ef-2f0bdb09801f service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:53.695212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3235be5-732a-4e8f-a6b7-66afff30110d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:53.695913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3235be5-732a-4e8f-a6b7-66afff30110d service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:53.698015 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3235be5-732a-4e8f-a6b7-66afff30110d service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:53.698307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3235be5-732a-4e8f-a6b7-66afff30110d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:53.719084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3235be5-732a-4e8f-a6b7-66afff30110d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:53.720872 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3235be5-732a-4e8f-a6b7-66afff30110d service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:53.750193 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 834/1645] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:28:53 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:28:54.006366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9ae0f-957a-4f13-a714-084ea821f632 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:54.018934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9ae0f-957a-4f13-a714-084ea821f632 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:54.028879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287e8c5-5fd0-47ed-8344-924975a0642e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:54.029620 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287e8c5-5fd0-47ed-8344-924975a0642e service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:54.029948 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287e8c5-5fd0-47ed-8344-924975a0642e service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:54.030303 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287e8c5-5fd0-47ed-8344-924975a0642e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:54.168988 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e287e8c5-5fd0-47ed-8344-924975a0642e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:54.170826 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e287e8c5-5fd0-47ed-8344-924975a0642e service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:54.202987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 812/1646] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:28:54 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:28:54.376337 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcbdab-401a-49a1-abf5-d6acfb05daac None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:54.376698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcbdab-401a-49a1-abf5-d6acfb05daac None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:54.376954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcbdab-401a-49a1-abf5-d6acfb05daac None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:54.379612 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 835/1647] 10.222.0.33 () {54 vars in 962 bytes} [Mon Jul 18 10:28:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:28:54.395056 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287e8c5-5fd0-47ed-8344-924975a0642e service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:54.398112 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb2afd-6d3c-4bee-98dd-68ca678aca1b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:54.398508 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb2afd-6d3c-4bee-98dd-68ca678aca1b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:54.398790 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb2afd-6d3c-4bee-98dd-68ca678aca1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:54.438005 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-03eb2afd-6d3c-4bee-98dd-68ca678aca1b None None] MFA Rules not processed for user `29048c2021e340b081ae44a390583e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:54.439748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-03eb2afd-6d3c-4bee-98dd-68ca678aca1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:54.456244 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 836/1648] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Jul 18 10:28:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:54.463944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c823ac8-af87-4743-896c-65f5481a1062 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:54.464264 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c823ac8-af87-4743-896c-65f5481a1062 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:54.464430 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c823ac8-af87-4743-896c-65f5481a1062 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:54.466387 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 837/1649] 10.222.0.33 () {54 vars in 912 bytes} [Mon Jul 18 10:28:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:28:54.473316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dd77e-f7c5-491d-93e1-756d5dd3793a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:54.475764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dd77e-f7c5-491d-93e1-756d5dd3793a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:54.476238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dd77e-f7c5-491d-93e1-756d5dd3793a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:54.531146 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-da8dd77e-f7c5-491d-93e1-756d5dd3793a None None] MFA Rules not processed for user `29048c2021e340b081ae44a390583e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:54.532727 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-da8dd77e-f7c5-491d-93e1-756d5dd3793a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:54.553590 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 838/1650] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 18 10:28:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:28:54.574140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8dd77e-f7c5-491d-93e1-756d5dd3793a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:54.585899 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8dd77e-f7c5-491d-93e1-756d5dd3793a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:54.592919 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287e8c5-5fd0-47ed-8344-924975a0642e service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:54.594707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc6a1e-0e0a-4927-a842-752fa1a0be31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:54.595418 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc6a1e-0e0a-4927-a842-752fa1a0be31 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:54.596528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc6a1e-0e0a-4927-a842-752fa1a0be31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:54.598247 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc6a1e-0e0a-4927-a842-752fa1a0be31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:54.609330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ec074-07f6-4907-92f2-dde93bb16a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:54.612688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ec074-07f6-4907-92f2-dde93bb16a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:54.613465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ec074-07f6-4907-92f2-dde93bb16a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:54.615377 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ec074-07f6-4907-92f2-dde93bb16a0f admin admin] PATH_INFO: `/v3/users/a6d07f10b21d483cb40ded0d83d74fa7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:54.619839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ec074-07f6-4907-92f2-dde93bb16a0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6d07f10b21d483cb40ded0d83d74fa7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:54.625082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-db8ec074-07f6-4907-92f2-dde93bb16a0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:28:54.625082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 18 10:28:54.625082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: Jul 18 10:28:54.627916 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ec074-07f6-4907-92f2-dde93bb16a0f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:54.635008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc6a1e-0e0a-4927-a842-752fa1a0be31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:54.636723 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc6a1e-0e0a-4927-a842-752fa1a0be31 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:54.678442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 839/1651] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:28:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:28:54.751480 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db8ec074-07f6-4907-92f2-dde93bb16a0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6d07f10b21d483cb40ded0d83d74fa7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:54.753420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db8ec074-07f6-4907-92f2-dde93bb16a0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6d07f10b21d483cb40ded0d83d74fa7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:54.754644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db8ec074-07f6-4907-92f2-dde93bb16a0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6d07f10b21d483cb40ded0d83d74fa7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:54.774531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db8ec074-07f6-4907-92f2-dde93bb16a0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6d07f10b21d483cb40ded0d83d74fa7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:54.802564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db8ec074-07f6-4907-92f2-dde93bb16a0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6d07f10b21d483cb40ded0d83d74fa7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:54.807924 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:54.830791 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 813/1652] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:28:54 2022] DELETE /identity/v3/users/a6d07f10b21d483cb40ded0d83d74fa7 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:54.836296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ab1de-a3fc-4f5f-be35-60a68030a8fa None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:54.836630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ab1de-a3fc-4f5f-be35-60a68030a8fa None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:54.836938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ab1de-a3fc-4f5f-be35-60a68030a8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:54.865310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9c0ab1de-a3fc-4f5f-be35-60a68030a8fa None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:54.866545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9c0ab1de-a3fc-4f5f-be35-60a68030a8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:54.953334 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:54.968614 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 840/1653] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:28:54.994298 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ec074-07f6-4907-92f2-dde93bb16a0f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:54.996149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-db8ec074-07f6-4907-92f2-dde93bb16a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:55.115360 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ec074-07f6-4907-92f2-dde93bb16a0f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:55.125754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70b3c4-a16d-46f7-b881-704ce98ce5a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:55.126446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70b3c4-a16d-46f7-b881-704ce98ce5a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:55.126734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70b3c4-a16d-46f7-b881-704ce98ce5a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:55.127042 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70b3c4-a16d-46f7-b881-704ce98ce5a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:55.150188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70b3c4-a16d-46f7-b881-704ce98ce5a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:55.151560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70b3c4-a16d-46f7-b881-704ce98ce5a5 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:55.175009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 814/1654] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:28:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:28:57.344695 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ab1de-a3fc-4f5f-be35-60a68030a8fa None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:57.346901 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9c0ab1de-a3fc-4f5f-be35-60a68030a8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:57.377341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ab1de-a3fc-4f5f-be35-60a68030a8fa None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:57.422450 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:57.426781 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:57.428094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:57.428837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f admin admin] PATH_INFO: `/v3/projects/f2fc95f0534240e1a13971da652442ff` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:57.437071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2fc95f0534240e1a13971da652442ff)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:57.438362 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 10:28:57.438362 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Jul 18 10:28:57.438362 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: Jul 18 10:28:57.444793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:57.612114 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2fc95f0534240e1a13971da652442ff', 'request_id': 'req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:57.672492 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2fc95f0534240e1a13971da652442ff', 'request_id': 'req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:57.697238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2fc95f0534240e1a13971da652442ff', 'request_id': 'req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:57.727358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f admin admin] The token cache is being invalidate because project f2fc95f0534240e1a13971da652442ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:28:57.727930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:57.728654 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:57.731109 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 841/1655] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:28:57 2022] DELETE /identity/v3/projects/f2fc95f0534240e1a13971da652442ff => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:57.734880 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70b3c4-a16d-46f7-b881-704ce98ce5a5 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:57.747088 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70b3c4-a16d-46f7-b881-704ce98ce5a5 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:57.862981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b63c7d-ceab-4ee4-8e7a-d3d061a4fea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:57.864137 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b63c7d-ceab-4ee4-8e7a-d3d061a4fea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:57.864762 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b63c7d-ceab-4ee4-8e7a-d3d061a4fea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:57.865230 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b63c7d-ceab-4ee4-8e7a-d3d061a4fea3 admin admin] PATH_INFO: `/v3/users/cb1b85f84b324a7eaf8266c9ef8108a8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:57.867029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b63c7d-ceab-4ee4-8e7a-d3d061a4fea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb1b85f84b324a7eaf8266c9ef8108a8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:57.869543 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b63c7d-ceab-4ee4-8e7a-d3d061a4fea3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:57.943362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-91b63c7d-ceab-4ee4-8e7a-d3d061a4fea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb1b85f84b324a7eaf8266c9ef8108a8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:57.944783 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-91b63c7d-ceab-4ee4-8e7a-d3d061a4fea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb1b85f84b324a7eaf8266c9ef8108a8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:57.945606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-91b63c7d-ceab-4ee4-8e7a-d3d061a4fea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb1b85f84b324a7eaf8266c9ef8108a8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:57.954959 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-91b63c7d-ceab-4ee4-8e7a-d3d061a4fea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb1b85f84b324a7eaf8266c9ef8108a8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:57.970069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-91b63c7d-ceab-4ee4-8e7a-d3d061a4fea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb1b85f84b324a7eaf8266c9ef8108a8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:57.999634 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 815/1656] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:28:57 2022] DELETE /identity/v3/users/cb1b85f84b324a7eaf8266c9ef8108a8 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:59.495979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:59.498117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:59.539693 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef222f-26ff-4207-b2ff-cbf4c2afe81f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:28:59.686462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6fde9-984f-45a3-a21a-97f1f181a63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:28:59.687816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6fde9-984f-45a3-a21a-97f1f181a63f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:59.688319 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6fde9-984f-45a3-a21a-97f1f181a63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:59.688888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6fde9-984f-45a3-a21a-97f1f181a63f admin admin] PATH_INFO: `/v3/projects/f18232497e5c4f37beb904eca31303f8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:59.691625 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6fde9-984f-45a3-a21a-97f1f181a63f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f18232497e5c4f37beb904eca31303f8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:28:59.696600 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6fde9-984f-45a3-a21a-97f1f181a63f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:28:59.809735 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-79e6fde9-984f-45a3-a21a-97f1f181a63f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f18232497e5c4f37beb904eca31303f8', 'request_id': 'req-79e6fde9-984f-45a3-a21a-97f1f181a63f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:59.830887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-79e6fde9-984f-45a3-a21a-97f1f181a63f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f18232497e5c4f37beb904eca31303f8', 'request_id': 'req-79e6fde9-984f-45a3-a21a-97f1f181a63f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:59.839050 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b63c7d-ceab-4ee4-8e7a-d3d061a4fea3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:28:59.841334 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-91b63c7d-ceab-4ee4-8e7a-d3d061a4fea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:59.851909 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-79e6fde9-984f-45a3-a21a-97f1f181a63f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f18232497e5c4f37beb904eca31303f8', 'request_id': 'req-79e6fde9-984f-45a3-a21a-97f1f181a63f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:59.888725 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-79e6fde9-984f-45a3-a21a-97f1f181a63f admin admin] The token cache is being invalidate because project f18232497e5c4f37beb904eca31303f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:28:59.889141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-79e6fde9-984f-45a3-a21a-97f1f181a63f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:59.889986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-79e6fde9-984f-45a3-a21a-97f1f181a63f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:28:59.893365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 842/1657] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:28:59 2022] DELETE /identity/v3/projects/f18232497e5c4f37beb904eca31303f8 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:28:59.902897 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:28:59.909832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390a0ec-656a-449a-ae29-5e7b2a937325 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:28:59.910349 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390a0ec-656a-449a-ae29-5e7b2a937325 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:28:59.912185 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390a0ec-656a-449a-ae29-5e7b2a937325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:28:59.957841 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4390a0ec-656a-449a-ae29-5e7b2a937325 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:28:59.959834 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4390a0ec-656a-449a-ae29-5e7b2a937325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:28:59.988866 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:00.028027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b63c7d-ceab-4ee4-8e7a-d3d061a4fea3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:00.085993 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 843/1658] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:00.093490 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ed382-1585-46c3-bddf-a2c78d665282 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:00.094225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ed382-1585-46c3-bddf-a2c78d665282 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:00.094733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ed382-1585-46c3-bddf-a2c78d665282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:00.144488 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f99ed382-1585-46c3-bddf-a2c78d665282 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:00.146866 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f99ed382-1585-46c3-bddf-a2c78d665282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:00.149180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95b3c1-b3f3-453a-9329-978b5088fc54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:00.149867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95b3c1-b3f3-453a-9329-978b5088fc54 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:00.150154 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95b3c1-b3f3-453a-9329-978b5088fc54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:00.150425 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95b3c1-b3f3-453a-9329-978b5088fc54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:00.158647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7f95b3c1-b3f3-453a-9329-978b5088fc54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:00.173655 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 844/1659] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:00.177942 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7fd44-3de7-4630-8010-86a55cabdecc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:00.178245 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7fd44-3de7-4630-8010-86a55cabdecc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:00.181682 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7fd44-3de7-4630-8010-86a55cabdecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:00.267708 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a1f7fd44-3de7-4630-8010-86a55cabdecc None None] MFA Rules not processed for user `f25c72896a2d4d619e5ad4b5a0a79aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:00.270252 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a1f7fd44-3de7-4630-8010-86a55cabdecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:00.319860 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95b3c1-b3f3-453a-9329-978b5088fc54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:00.323038 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95b3c1-b3f3-453a-9329-978b5088fc54 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:00.348686 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 816/1660] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:28:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:29:00.351081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95b3c1-b3f3-453a-9329-978b5088fc54 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:00.387035 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95b3c1-b3f3-453a-9329-978b5088fc54 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:00.397637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc7c39-0130-4a99-9b78-2a418662d2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:00.398455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc7c39-0130-4a99-9b78-2a418662d2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:00.399667 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc7c39-0130-4a99-9b78-2a418662d2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:00.400281 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc7c39-0130-4a99-9b78-2a418662d2e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:00.402631 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bc7c39-0130-4a99-9b78-2a418662d2e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:00.403189 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bc7c39-0130-4a99-9b78-2a418662d2e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:00.407521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bc7c39-0130-4a99-9b78-2a418662d2e5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:00.438071 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 817/1661] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:00.443866 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc7c39-0130-4a99-9b78-2a418662d2e5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:00.474467 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 845/1662] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:00.474558 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc7c39-0130-4a99-9b78-2a418662d2e5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:00.484369 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7fd44-3de7-4630-8010-86a55cabdecc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:00.486210 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a1f7fd44-3de7-4630-8010-86a55cabdecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:00.529672 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f0291-eaa5-4776-a62f-d7a831199054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:00.530362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f0291-eaa5-4776-a62f-d7a831199054 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:00.532581 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f0291-eaa5-4776-a62f-d7a831199054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:00.532876 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f0291-eaa5-4776-a62f-d7a831199054 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:00.533656 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f0291-eaa5-4776-a62f-d7a831199054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:00.535631 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f0291-eaa5-4776-a62f-d7a831199054 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:00.591546 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 818/1663] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:00 2022] POST /identity/v3/projects => generated 369 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:00.592468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f0291-eaa5-4776-a62f-d7a831199054 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:00.713936 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7fd44-3de7-4630-8010-86a55cabdecc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:00.724078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:00.761318 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f0291-eaa5-4776-a62f-d7a831199054 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:00.776700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fae5bd-a2ca-416d-ad9c-7173d93cfb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:00.778050 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fae5bd-a2ca-416d-ad9c-7173d93cfb43 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:00.778383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fae5bd-a2ca-416d-ad9c-7173d93cfb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:00.778725 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fae5bd-a2ca-416d-ad9c-7173d93cfb43 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:00.779439 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fae5bd-a2ca-416d-ad9c-7173d93cfb43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:00.781877 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fae5bd-a2ca-416d-ad9c-7173d93cfb43 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:00.808414 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:00.829525 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 819/1664] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:00 2022] POST /identity/v3/users => generated 285 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:00.837081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fae5bd-a2ca-416d-ad9c-7173d93cfb43 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:00.865645 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fae5bd-a2ca-416d-ad9c-7173d93cfb43 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:00.887387 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54b270-49c3-4158-8330-d1faa3c56405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:00.888254 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54b270-49c3-4158-8330-d1faa3c56405 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:00.888706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54b270-49c3-4158-8330-d1faa3c56405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:00.889152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54b270-49c3-4158-8330-d1faa3c56405 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:00.890198 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54b270-49c3-4158-8330-d1faa3c56405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:00.895101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54b270-49c3-4158-8330-d1faa3c56405 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:00.901761 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a4b5f-c58b-42e3-92f0-1ce54100742b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:00.903934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a4b5f-c58b-42e3-92f0-1ce54100742b service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:00.904224 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a4b5f-c58b-42e3-92f0-1ce54100742b service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:00.904581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a4b5f-c58b-42e3-92f0-1ce54100742b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:00.939127 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 820/1665] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:29:00.940340 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54b270-49c3-4158-8330-d1faa3c56405 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:00.963149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54b270-49c3-4158-8330-d1faa3c56405 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:00.978283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52b23e-c29b-44f8-af98-20c2ce024c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:00.979524 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52b23e-c29b-44f8-af98-20c2ce024c62 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:00.980010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52b23e-c29b-44f8-af98-20c2ce024c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:00.980673 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52b23e-c29b-44f8-af98-20c2ce024c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:00.981792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52b23e-c29b-44f8-af98-20c2ce024c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:00.987579 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52b23e-c29b-44f8-af98-20c2ce024c62 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:01.038826 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 821/1666] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:01.042005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52b23e-c29b-44f8-af98-20c2ce024c62 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:01.069005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52b23e-c29b-44f8-af98-20c2ce024c62 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:01.084638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b17b8-6a14-4bb8-b72f-8a50971e8331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:01.085729 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b17b8-6a14-4bb8-b72f-8a50971e8331 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:01.086082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b17b8-6a14-4bb8-b72f-8a50971e8331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:01.086362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b17b8-6a14-4bb8-b72f-8a50971e8331 admin admin] PATH_INFO: `/v3/projects/a6f03dceff3e4dc2be84fc41202d2c17/users/213bad94d3a44f06b0c9e8bdd54475ce/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:01.113089 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a4b5f-c58b-42e3-92f0-1ce54100742b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:01.117409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a4b5f-c58b-42e3-92f0-1ce54100742b service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:01.123370 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b17b8-6a14-4bb8-b72f-8a50971e8331 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6f03dceff3e4dc2be84fc41202d2c17, user_id=213bad94d3a44f06b0c9e8bdd54475ce, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:01.130874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b17b8-6a14-4bb8-b72f-8a50971e8331 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:01.168393 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 846/1667] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 685 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:01.189623 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 822/1668] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:01 2022] PUT /identity/v3/projects/a6f03dceff3e4dc2be84fc41202d2c17/users/213bad94d3a44f06b0c9e8bdd54475ce/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:01.206453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861fc71-5d32-4ae8-90e2-9ce80c407703 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:01.206845 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861fc71-5d32-4ae8-90e2-9ce80c407703 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:01.207135 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861fc71-5d32-4ae8-90e2-9ce80c407703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:01.281096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0861fc71-5d32-4ae8-90e2-9ce80c407703 None None] MFA Rules not processed for user `213bad94d3a44f06b0c9e8bdd54475ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:01.283336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0861fc71-5d32-4ae8-90e2-9ce80c407703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:01.441319 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 847/1669] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:01 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:01.446722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b17b8-6a14-4bb8-b72f-8a50971e8331 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:01.594185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b17b8-6a14-4bb8-b72f-8a50971e8331 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:01.601387 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb609a2-d9d8-4a1c-a6fd-612cf6aa0a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:01.602572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb609a2-d9d8-4a1c-a6fd-612cf6aa0a2e admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:01.603001 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb609a2-d9d8-4a1c-a6fd-612cf6aa0a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:01.603501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb609a2-d9d8-4a1c-a6fd-612cf6aa0a2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:01.606740 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb609a2-d9d8-4a1c-a6fd-612cf6aa0a2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:01.613373 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb609a2-d9d8-4a1c-a6fd-612cf6aa0a2e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:01.687026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 823/1670] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:01 2022] POST /identity/v3/projects => generated 369 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:01.690950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861fc71-5d32-4ae8-90e2-9ce80c407703 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:01.709477 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861fc71-5d32-4ae8-90e2-9ce80c407703 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:01.879256 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a7d31-8741-4780-a4d6-83c26ee5e6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:01.883288 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a7d31-8741-4780-a4d6-83c26ee5e6f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:01.883753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a7d31-8741-4780-a4d6-83c26ee5e6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:01.884043 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a7d31-8741-4780-a4d6-83c26ee5e6f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:01.884873 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a7d31-8741-4780-a4d6-83c26ee5e6f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:01.886846 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a7d31-8741-4780-a4d6-83c26ee5e6f1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:01.908898 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:01.937259 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 848/1671] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:01 2022] POST /identity/v3/users => generated 291 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:01.938529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb609a2-d9d8-4a1c-a6fd-612cf6aa0a2e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:01.952528 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb609a2-d9d8-4a1c-a6fd-612cf6aa0a2e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:01.965553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9194a22-cb40-480d-91d8-3937a6c03a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:01.967520 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9194a22-cb40-480d-91d8-3937a6c03a79 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:01.967819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9194a22-cb40-480d-91d8-3937a6c03a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:01.972344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9194a22-cb40-480d-91d8-3937a6c03a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:01.973322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9194a22-cb40-480d-91d8-3937a6c03a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:01.978163 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9194a22-cb40-480d-91d8-3937a6c03a79 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:02.038867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 824/1672] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:02.049400 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a7d31-8741-4780-a4d6-83c26ee5e6f1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:02.082887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a7d31-8741-4780-a4d6-83c26ee5e6f1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:02.103102 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba14d8-9f18-4538-a5f0-e39edd21504c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:02.104301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba14d8-9f18-4538-a5f0-e39edd21504c admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:02.104640 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba14d8-9f18-4538-a5f0-e39edd21504c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:02.107843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba14d8-9f18-4538-a5f0-e39edd21504c admin admin] PATH_INFO: `/v3/projects/75505338a2fa4c548fb1e464464ae9b8/users/42e5875ee4f04d668930b5198ed14a4f/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:02.214760 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ba14d8-9f18-4538-a5f0-e39edd21504c admin admin] RBAC: Authorizing `identity:create_grant(project_id=75505338a2fa4c548fb1e464464ae9b8, user_id=42e5875ee4f04d668930b5198ed14a4f, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:02.223242 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ba14d8-9f18-4538-a5f0-e39edd21504c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:02.291402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 849/1673] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:02 2022] PUT /identity/v3/projects/75505338a2fa4c548fb1e464464ae9b8/users/42e5875ee4f04d668930b5198ed14a4f/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:02.295955 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9194a22-cb40-480d-91d8-3937a6c03a79 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:02.318015 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9194a22-cb40-480d-91d8-3937a6c03a79 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:02.474485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f1753-ae5e-4d2c-839b-0f71486e86bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:02.475284 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f1753-ae5e-4d2c-839b-0f71486e86bb admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:02.475911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f1753-ae5e-4d2c-839b-0f71486e86bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:02.475911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f1753-ae5e-4d2c-839b-0f71486e86bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:02.476864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f1753-ae5e-4d2c-839b-0f71486e86bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:02.478575 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f1753-ae5e-4d2c-839b-0f71486e86bb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:02.511874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 825/1674] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:02.513275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba14d8-9f18-4538-a5f0-e39edd21504c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:02.527724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba14d8-9f18-4538-a5f0-e39edd21504c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:02.537358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a4218-fd9a-49c9-a492-d2c1e5ce0ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:02.538350 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a4218-fd9a-49c9-a492-d2c1e5ce0ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:02.538640 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a4218-fd9a-49c9-a492-d2c1e5ce0ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:02.538990 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a4218-fd9a-49c9-a492-d2c1e5ce0ce7 admin admin] PATH_INFO: `/v3/projects/75505338a2fa4c548fb1e464464ae9b8/users/42e5875ee4f04d668930b5198ed14a4f/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:02.550475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a4218-fd9a-49c9-a492-d2c1e5ce0ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75505338a2fa4c548fb1e464464ae9b8, user_id=42e5875ee4f04d668930b5198ed14a4f, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:02.556295 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a4218-fd9a-49c9-a492-d2c1e5ce0ce7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:02.593074 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 850/1675] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:02 2022] PUT /identity/v3/projects/75505338a2fa4c548fb1e464464ae9b8/users/42e5875ee4f04d668930b5198ed14a4f/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:02.604486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4952d-3ae8-4830-b7b0-cf0bae531c40 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:02.604884 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4952d-3ae8-4830-b7b0-cf0bae531c40 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:02.605547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4952d-3ae8-4830-b7b0-cf0bae531c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:02.611527 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:02.663103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-97f4952d-3ae8-4830-b7b0-cf0bae531c40 None None] MFA Rules not processed for user `42e5875ee4f04d668930b5198ed14a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:02.664555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-97f4952d-3ae8-4830-b7b0-cf0bae531c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:02.856326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 826/1676] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:02 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:02.870414 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a4218-fd9a-49c9-a492-d2c1e5ce0ce7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:02.871779 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e80a4218-fd9a-49c9-a492-d2c1e5ce0ce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:02.999462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a4218-fd9a-49c9-a492-d2c1e5ce0ce7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:03.004821 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff760bf3-28a9-4c1c-9550-6e636436a89c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:03.005653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff760bf3-28a9-4c1c-9550-6e636436a89c service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:03.006099 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff760bf3-28a9-4c1c-9550-6e636436a89c service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:03.006518 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff760bf3-28a9-4c1c-9550-6e636436a89c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:03.029780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff760bf3-28a9-4c1c-9550-6e636436a89c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:03.031653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff760bf3-28a9-4c1c-9550-6e636436a89c service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:03.058186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 851/1677] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:02 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:04.044163 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc36ae4-dbd1-44db-b77a-7d0946b309d9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:04.045113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc36ae4-dbd1-44db-b77a-7d0946b309d9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:04.045635 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc36ae4-dbd1-44db-b77a-7d0946b309d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:04.090548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7cc36ae4-dbd1-44db-b77a-7d0946b309d9 None None] MFA Rules not processed for user `72a431c9708f4694bc0d514133e76088`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:04.092492 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7cc36ae4-dbd1-44db-b77a-7d0946b309d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:04.216720 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 827/1678] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:04.218363 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255c7bb-97e2-4ddd-84ca-b18a61fde7da None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:04.218795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255c7bb-97e2-4ddd-84ca-b18a61fde7da None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:04.222385 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255c7bb-97e2-4ddd-84ca-b18a61fde7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:04.250414 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a255c7bb-97e2-4ddd-84ca-b18a61fde7da None None] MFA Rules not processed for user `72a431c9708f4694bc0d514133e76088`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:04.255256 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a255c7bb-97e2-4ddd-84ca-b18a61fde7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:04.269054 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 852/1679] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:29:04.277105 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc36ae4-dbd1-44db-b77a-7d0946b309d9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:04.278545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7cc36ae4-dbd1-44db-b77a-7d0946b309d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:04.429352 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc36ae4-dbd1-44db-b77a-7d0946b309d9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:04.440701 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb379deb-cc22-43c8-96f2-7649111cb187 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:04.447063 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb379deb-cc22-43c8-96f2-7649111cb187 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:04.448585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb379deb-cc22-43c8-96f2-7649111cb187 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:04.449245 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb379deb-cc22-43c8-96f2-7649111cb187 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:04.503159 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb379deb-cc22-43c8-96f2-7649111cb187 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:04.505207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb379deb-cc22-43c8-96f2-7649111cb187 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:04.567768 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 828/1680] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:05.727619 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7439b5-1c2c-4156-b789-d6763ec4f1f4 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:05.728078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7439b5-1c2c-4156-b789-d6763ec4f1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:05.728183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7439b5-1c2c-4156-b789-d6763ec4f1f4 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:05.740203 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 853/1681] 10.222.0.33 () {54 vars in 912 bytes} [Mon Jul 18 10:29:05 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:29:05.766939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bab35-f108-49b0-8147-0763fa3c6a42 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:05.767379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bab35-f108-49b0-8147-0763fa3c6a42 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:05.767600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bab35-f108-49b0-8147-0763fa3c6a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:05.774266 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:05.879697 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c26bab35-f108-49b0-8147-0763fa3c6a42 None None] MFA Rules not processed for user `10d29b7934374f34923587eda9be0ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:05.881955 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c26bab35-f108-49b0-8147-0763fa3c6a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:06.065855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 829/1682] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 18 10:29:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:06.071474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7439b5-1c2c-4156-b789-d6763ec4f1f4 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:06.074014 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6f7439b5-1c2c-4156-b789-d6763ec4f1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:06.119251 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7439b5-1c2c-4156-b789-d6763ec4f1f4 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:06.134172 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e08aa-2de8-4555-8486-6d983db2351d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:06.134890 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e08aa-2de8-4555-8486-6d983db2351d service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:06.135170 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e08aa-2de8-4555-8486-6d983db2351d service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:06.135448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e08aa-2de8-4555-8486-6d983db2351d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:06.147767 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e08aa-2de8-4555-8486-6d983db2351d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:06.149278 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e08aa-2de8-4555-8486-6d983db2351d service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:06.183474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 854/1683] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:06.679026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245e45d-b161-4fec-b80a-71f4f76cfd1e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:06.679026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245e45d-b161-4fec-b80a-71f4f76cfd1e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:06.688155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245e45d-b161-4fec-b80a-71f4f76cfd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:06.816338 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8245e45d-b161-4fec-b80a-71f4f76cfd1e None None] MFA Rules not processed for user `986f90dfbad14324b3fffbd790d60a37`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:06.821232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8245e45d-b161-4fec-b80a-71f4f76cfd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:07.037696 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 830/1684] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:07.057147 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e08aa-2de8-4555-8486-6d983db2351d service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:07.067250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245e45d-b161-4fec-b80a-71f4f76cfd1e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:07.110658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245e45d-b161-4fec-b80a-71f4f76cfd1e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:07.132658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4ae41-08fb-46db-b867-82d70fe44cd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:07.134492 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4ae41-08fb-46db-b867-82d70fe44cd5 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:07.136550 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4ae41-08fb-46db-b867-82d70fe44cd5 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:07.137888 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4ae41-08fb-46db-b867-82d70fe44cd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:07.161564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4ae41-08fb-46db-b867-82d70fe44cd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:07.163824 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4ae41-08fb-46db-b867-82d70fe44cd5 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:07.216923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 831/1685] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:07.270956 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626d0bb-05d8-436c-ae2a-60f343bdb6cb None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:07.271500 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626d0bb-05d8-436c-ae2a-60f343bdb6cb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:07.272532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626d0bb-05d8-436c-ae2a-60f343bdb6cb None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:07.274612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 832/1686] 10.222.0.33 () {54 vars in 961 bytes} [Mon Jul 18 10:29:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:29:07.285386 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a00319-325c-4ada-9f8b-784c289b6f16 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:07.285827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a00319-325c-4ada-9f8b-784c289b6f16 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:07.286558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e08aa-2de8-4555-8486-6d983db2351d service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:07.286989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a00319-325c-4ada-9f8b-784c289b6f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:07.311978 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c50e58-f900-429d-b50b-9d01ca20c396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:07.313521 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c50e58-f900-429d-b50b-9d01ca20c396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:07.313807 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c50e58-f900-429d-b50b-9d01ca20c396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:07.314248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c50e58-f900-429d-b50b-9d01ca20c396 admin admin] PATH_INFO: `/v3/users/213bad94d3a44f06b0c9e8bdd54475ce` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:07.317166 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c50e58-f900-429d-b50b-9d01ca20c396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=213bad94d3a44f06b0c9e8bdd54475ce)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:07.329034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c50e58-f900-429d-b50b-9d01ca20c396 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:07.352827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c2a00319-325c-4ada-9f8b-784c289b6f16 None None] MFA Rules not processed for user `d07165f554a445169b2c3f0d173c0bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:07.356095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c2a00319-325c-4ada-9f8b-784c289b6f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:07.441337 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:07.489373 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-60c50e58-f900-429d-b50b-9d01ca20c396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '213bad94d3a44f06b0c9e8bdd54475ce'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:07.530721 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-60c50e58-f900-429d-b50b-9d01ca20c396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '213bad94d3a44f06b0c9e8bdd54475ce'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:07.533934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-60c50e58-f900-429d-b50b-9d01ca20c396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '213bad94d3a44f06b0c9e8bdd54475ce'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:07.534781 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-60c50e58-f900-429d-b50b-9d01ca20c396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '213bad94d3a44f06b0c9e8bdd54475ce'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:07.555993 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-60c50e58-f900-429d-b50b-9d01ca20c396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '213bad94d3a44f06b0c9e8bdd54475ce'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:07.595550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 855/1687] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:07 2022] DELETE /identity/v3/users/213bad94d3a44f06b0c9e8bdd54475ce => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:07.600375 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1f5f1-3c9d-486b-b3e7-4c1a05a3eab8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:07.600873 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1f5f1-3c9d-486b-b3e7-4c1a05a3eab8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:07.602773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1f5f1-3c9d-486b-b3e7-4c1a05a3eab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:07.653365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7ec1f5f1-3c9d-486b-b3e7-4c1a05a3eab8 None None] MFA Rules not processed for user `72a431c9708f4694bc0d514133e76088`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:07.661871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7ec1f5f1-3c9d-486b-b3e7-4c1a05a3eab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:07.680832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 833/1688] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Jul 18 10:29:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:07.697457 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3a3da-bf89-444d-944e-86f78ce473a8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:07.698072 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3a3da-bf89-444d-944e-86f78ce473a8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:07.698832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3a3da-bf89-444d-944e-86f78ce473a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:07.755241 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-99e3a3da-bf89-444d-944e-86f78ce473a8 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:07.756642 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-99e3a3da-bf89-444d-944e-86f78ce473a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:07.763174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:07.809943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 856/1689] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:07.811024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1f5f1-3c9d-486b-b3e7-4c1a05a3eab8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:07.883055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 834/1690] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:07.903805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3a3da-bf89-444d-944e-86f78ce473a8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:07.905577 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-99e3a3da-bf89-444d-944e-86f78ce473a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:07.954478 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1f5f1-3c9d-486b-b3e7-4c1a05a3eab8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:07.968647 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7413c17-f84c-4e7e-a86a-dd57477a98b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd07165f554a445169b2c3f0d173c0bb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:07.969299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7413c17-f84c-4e7e-a86a-dd57477a98b2 service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:07.969607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7413c17-f84c-4e7e-a86a-dd57477a98b2 service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:07.969848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7413c17-f84c-4e7e-a86a-dd57477a98b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:07.974583 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a7413c17-f84c-4e7e-a86a-dd57477a98b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:07.989999 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7413c17-f84c-4e7e-a86a-dd57477a98b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:07.993295 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7413c17-f84c-4e7e-a86a-dd57477a98b2 service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:08.006785 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 857/1691] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:08.071282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3a3da-bf89-444d-944e-86f78ce473a8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:08.083639 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67fa87-be8b-47b2-bb6f-443ee9be5eaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:08.084798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67fa87-be8b-47b2-bb6f-443ee9be5eaa service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:08.085296 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67fa87-be8b-47b2-bb6f-443ee9be5eaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:08.085775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67fa87-be8b-47b2-bb6f-443ee9be5eaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:08.102117 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67fa87-be8b-47b2-bb6f-443ee9be5eaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:08.104195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67fa87-be8b-47b2-bb6f-443ee9be5eaa service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:08.116945 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 835/1692] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:08.394023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19c540-0c1b-4f7a-aaf9-4d8e374e20d9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:08.394422 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19c540-0c1b-4f7a-aaf9-4d8e374e20d9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:08.394580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19c540-0c1b-4f7a-aaf9-4d8e374e20d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:08.425981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0b19c540-0c1b-4f7a-aaf9-4d8e374e20d9 None None] MFA Rules not processed for user `f25c72896a2d4d619e5ad4b5a0a79aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:08.427781 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0b19c540-0c1b-4f7a-aaf9-4d8e374e20d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:08.551273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:08.579843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 858/1693] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:08.796109 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e35c0e-f388-4685-8e72-dbb3021c9ff5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:08.796403 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e35c0e-f388-4685-8e72-dbb3021c9ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:08.796713 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e35c0e-f388-4685-8e72-dbb3021c9ff5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:08.798169 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 836/1694] 10.222.0.33 () {54 vars in 912 bytes} [Mon Jul 18 10:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:29:08.808331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478eabb2-93d7-4a79-9185-8554f0d0ea83 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:08.808707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478eabb2-93d7-4a79-9185-8554f0d0ea83 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:08.808993 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478eabb2-93d7-4a79-9185-8554f0d0ea83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:08.857334 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-478eabb2-93d7-4a79-9185-8554f0d0ea83 None None] MFA Rules not processed for user `10d29b7934374f34923587eda9be0ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:08.858799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-478eabb2-93d7-4a79-9185-8554f0d0ea83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:08.868293 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 859/1695] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 18 10:29:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:08.921180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e35c0e-f388-4685-8e72-dbb3021c9ff5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:08.939702 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e35c0e-f388-4685-8e72-dbb3021c9ff5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:08.951077 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c8a10-e54b-4a6c-81e0-0c8ffe63dd31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:08.954508 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c8a10-e54b-4a6c-81e0-0c8ffe63dd31 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:08.955587 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c8a10-e54b-4a6c-81e0-0c8ffe63dd31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:08.958572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c8a10-e54b-4a6c-81e0-0c8ffe63dd31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:08.976568 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c8a10-e54b-4a6c-81e0-0c8ffe63dd31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:08.977331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c8a10-e54b-4a6c-81e0-0c8ffe63dd31 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:08.996653 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 837/1696] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:09.569182 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478eabb2-93d7-4a79-9185-8554f0d0ea83 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:09.572265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-478eabb2-93d7-4a79-9185-8554f0d0ea83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:09.605324 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa6787-18ac-4c85-b370-289379441212 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:09.605778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa6787-18ac-4c85-b370-289379441212 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:09.606065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa6787-18ac-4c85-b370-289379441212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:09.613743 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478eabb2-93d7-4a79-9185-8554f0d0ea83 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:09.629346 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb3b27-fb93-48cb-83a1-80663483e05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:09.631167 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb3b27-fb93-48cb-83a1-80663483e05d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:09.631981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb3b27-fb93-48cb-83a1-80663483e05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:09.632282 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb3b27-fb93-48cb-83a1-80663483e05d admin admin] PATH_INFO: `/v3/projects/a6f03dceff3e4dc2be84fc41202d2c17` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:09.637599 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb3b27-fb93-48cb-83a1-80663483e05d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6f03dceff3e4dc2be84fc41202d2c17)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:09.640000 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb3b27-fb93-48cb-83a1-80663483e05d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:09.664205 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-62fa6787-18ac-4c85-b370-289379441212 None None] MFA Rules not processed for user `c0f993e44c624f65abae22540ce7dffc`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:09.665483 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-62fa6787-18ac-4c85-b370-289379441212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:09.757341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-97eb3b27-fb93-48cb-83a1-80663483e05d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6f03dceff3e4dc2be84fc41202d2c17', 'request_id': 'req-97eb3b27-fb93-48cb-83a1-80663483e05d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:09.772733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-97eb3b27-fb93-48cb-83a1-80663483e05d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6f03dceff3e4dc2be84fc41202d2c17', 'request_id': 'req-97eb3b27-fb93-48cb-83a1-80663483e05d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:09.787192 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-97eb3b27-fb93-48cb-83a1-80663483e05d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6f03dceff3e4dc2be84fc41202d2c17', 'request_id': 'req-97eb3b27-fb93-48cb-83a1-80663483e05d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:09.813908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 838/1697] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:09.823214 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-97eb3b27-fb93-48cb-83a1-80663483e05d admin admin] The token cache is being invalidate because project a6f03dceff3e4dc2be84fc41202d2c17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:09.823713 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-97eb3b27-fb93-48cb-83a1-80663483e05d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:09.828773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-97eb3b27-fb93-48cb-83a1-80663483e05d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:09.829773 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840027d-055c-472f-9def-ceb0429050d3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:09.830076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840027d-055c-472f-9def-ceb0429050d3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:09.830359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840027d-055c-472f-9def-ceb0429050d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:09.834618 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 860/1698] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:09 2022] DELETE /identity/v3/projects/a6f03dceff3e4dc2be84fc41202d2c17 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:09.835532 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb3b27-fb93-48cb-83a1-80663483e05d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:09.839356 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-97eb3b27-fb93-48cb-83a1-80663483e05d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:09.863477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b840027d-055c-472f-9def-ceb0429050d3 None None] MFA Rules not processed for user `c0f993e44c624f65abae22540ce7dffc`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:09.865602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b840027d-055c-472f-9def-ceb0429050d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:09.928403 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:10.001857 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 839/1699] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:10.002034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb3b27-fb93-48cb-83a1-80663483e05d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:10.017919 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89315cf3-66f0-4bb8-b50f-64b6f1199672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:10.018839 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89315cf3-66f0-4bb8-b50f-64b6f1199672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:10.019131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89315cf3-66f0-4bb8-b50f-64b6f1199672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:10.019413 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89315cf3-66f0-4bb8-b50f-64b6f1199672 admin admin] PATH_INFO: `/v3/users/42e5875ee4f04d668930b5198ed14a4f` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:10.021023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89315cf3-66f0-4bb8-b50f-64b6f1199672 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42e5875ee4f04d668930b5198ed14a4f)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:10.023565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89315cf3-66f0-4bb8-b50f-64b6f1199672 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:10.154947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-89315cf3-66f0-4bb8-b50f-64b6f1199672 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42e5875ee4f04d668930b5198ed14a4f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:10.183163 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-89315cf3-66f0-4bb8-b50f-64b6f1199672 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42e5875ee4f04d668930b5198ed14a4f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:10.186049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-89315cf3-66f0-4bb8-b50f-64b6f1199672 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42e5875ee4f04d668930b5198ed14a4f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:10.186808 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-89315cf3-66f0-4bb8-b50f-64b6f1199672 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42e5875ee4f04d668930b5198ed14a4f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:10.202433 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-89315cf3-66f0-4bb8-b50f-64b6f1199672 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42e5875ee4f04d668930b5198ed14a4f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:10.205423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:10.238057 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 861/1700] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:09 2022] DELETE /identity/v3/users/42e5875ee4f04d668930b5198ed14a4f => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:10.441675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840027d-055c-472f-9def-ceb0429050d3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:10.443444 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b840027d-055c-472f-9def-ceb0429050d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:10.713831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840027d-055c-472f-9def-ceb0429050d3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:10.916819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fe34e-4797-44b3-aa7e-a70936c13b22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:10.920570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fe34e-4797-44b3-aa7e-a70936c13b22 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:10.920987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fe34e-4797-44b3-aa7e-a70936c13b22 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:10.921351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fe34e-4797-44b3-aa7e-a70936c13b22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:10.930529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-877fe34e-4797-44b3-aa7e-a70936c13b22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:11.086864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fe34e-4797-44b3-aa7e-a70936c13b22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:11.090662 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fe34e-4797-44b3-aa7e-a70936c13b22 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:11.106929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce3d61-5289-46eb-91c3-63d1521a8e19 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:11.107819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce3d61-5289-46eb-91c3-63d1521a8e19 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:11.108306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce3d61-5289-46eb-91c3-63d1521a8e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:11.129370 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 840/1701] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 688 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:11.141543 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-abce3d61-5289-46eb-91c3-63d1521a8e19 None None] MFA Rules not processed for user `986f90dfbad14324b3fffbd790d60a37`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:11.144096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-abce3d61-5289-46eb-91c3-63d1521a8e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:11.162819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fe34e-4797-44b3-aa7e-a70936c13b22 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:11.164327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-877fe34e-4797-44b3-aa7e-a70936c13b22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:11.189922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:11.279610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 862/1702] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:11.333381 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fe34e-4797-44b3-aa7e-a70936c13b22 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:11.351787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f1f35-e0e5-47e1-8a4f-6f98c2ef36be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:11.352918 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f1f35-e0e5-47e1-8a4f-6f98c2ef36be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:11.353149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f1f35-e0e5-47e1-8a4f-6f98c2ef36be admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:11.353629 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f1f35-e0e5-47e1-8a4f-6f98c2ef36be admin admin] PATH_INFO: `/v3/projects/75505338a2fa4c548fb1e464464ae9b8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:11.355431 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f1f35-e0e5-47e1-8a4f-6f98c2ef36be admin admin] RBAC: Authorizing `identity:delete_project(project_id=75505338a2fa4c548fb1e464464ae9b8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:11.357601 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f1f35-e0e5-47e1-8a4f-6f98c2ef36be admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:11.468227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-265f1f35-e0e5-47e1-8a4f-6f98c2ef36be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75505338a2fa4c548fb1e464464ae9b8', 'request_id': 'req-265f1f35-e0e5-47e1-8a4f-6f98c2ef36be'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:11.489697 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-265f1f35-e0e5-47e1-8a4f-6f98c2ef36be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75505338a2fa4c548fb1e464464ae9b8', 'request_id': 'req-265f1f35-e0e5-47e1-8a4f-6f98c2ef36be'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:11.509476 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-265f1f35-e0e5-47e1-8a4f-6f98c2ef36be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75505338a2fa4c548fb1e464464ae9b8', 'request_id': 'req-265f1f35-e0e5-47e1-8a4f-6f98c2ef36be'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:11.532946 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-265f1f35-e0e5-47e1-8a4f-6f98c2ef36be admin admin] The token cache is being invalidate because project 75505338a2fa4c548fb1e464464ae9b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:11.533311 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-265f1f35-e0e5-47e1-8a4f-6f98c2ef36be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:11.533999 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-265f1f35-e0e5-47e1-8a4f-6f98c2ef36be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:11.557749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 841/1703] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:11 2022] DELETE /identity/v3/projects/75505338a2fa4c548fb1e464464ae9b8 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:11.561113 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcb24f-8699-477e-a6b1-7da155122c16 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:11.561649 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcb24f-8699-477e-a6b1-7da155122c16 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:11.561920 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcb24f-8699-477e-a6b1-7da155122c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:11.625889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-00bcb24f-8699-477e-a6b1-7da155122c16 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:11.631459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-00bcb24f-8699-477e-a6b1-7da155122c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:11.819404 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 863/1704] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:11.821386 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3345770-45a5-40b7-9289-8b8cc85d2c9b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:11.821684 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3345770-45a5-40b7-9289-8b8cc85d2c9b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:11.821972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3345770-45a5-40b7-9289-8b8cc85d2c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:11.863640 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c3345770-45a5-40b7-9289-8b8cc85d2c9b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:11.865763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c3345770-45a5-40b7-9289-8b8cc85d2c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:11.893957 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 842/1705] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:11.899398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcb24f-8699-477e-a6b1-7da155122c16 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:11.915329 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcb24f-8699-477e-a6b1-7da155122c16 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:11.929388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a4484-43d7-408a-b847-8f8cf298a233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:11.930304 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a4484-43d7-408a-b847-8f8cf298a233 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:11.930640 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a4484-43d7-408a-b847-8f8cf298a233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:11.931034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a4484-43d7-408a-b847-8f8cf298a233 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:11.931825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a4484-43d7-408a-b847-8f8cf298a233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:11.932141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a4484-43d7-408a-b847-8f8cf298a233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:11.936253 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a4484-43d7-408a-b847-8f8cf298a233 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:11.976810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 864/1706] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:11.977816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3345770-45a5-40b7-9289-8b8cc85d2c9b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:11.988220 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3345770-45a5-40b7-9289-8b8cc85d2c9b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:12.001214 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa5f6a-d383-418f-8226-bf14c82018e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:12.002220 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa5f6a-d383-418f-8226-bf14c82018e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:12.002646 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa5f6a-d383-418f-8226-bf14c82018e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:12.003089 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa5f6a-d383-418f-8226-bf14c82018e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:12.003928 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa5f6a-d383-418f-8226-bf14c82018e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:12.012661 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa5f6a-d383-418f-8226-bf14c82018e1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:12.057914 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 843/1707] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:11 2022] POST /identity/v3/projects => generated 383 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:29:12.062002 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a4484-43d7-408a-b847-8f8cf298a233 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:12.075954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a4484-43d7-408a-b847-8f8cf298a233 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:12.165474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d94ce1-c29e-4b86-95bd-4d1d1de901b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:12.167017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d94ce1-c29e-4b86-95bd-4d1d1de901b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:12.167491 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d94ce1-c29e-4b86-95bd-4d1d1de901b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:12.167954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d94ce1-c29e-4b86-95bd-4d1d1de901b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:12.169150 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d94ce1-c29e-4b86-95bd-4d1d1de901b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:12.171354 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d94ce1-c29e-4b86-95bd-4d1d1de901b5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:12.207808 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 865/1708] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:12 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:12.212004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa5f6a-d383-418f-8226-bf14c82018e1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:12.231104 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa5f6a-d383-418f-8226-bf14c82018e1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:12.237288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a13a2-04d2-435f-a11c-bb33fbcaf53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:12.238313 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a13a2-04d2-435f-a11c-bb33fbcaf53e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:12.238786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a13a2-04d2-435f-a11c-bb33fbcaf53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:12.241575 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a13a2-04d2-435f-a11c-bb33fbcaf53e admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:12.244009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a13a2-04d2-435f-a11c-bb33fbcaf53e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:12.246124 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a13a2-04d2-435f-a11c-bb33fbcaf53e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:12.268047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 844/1709] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:12.269306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d94ce1-c29e-4b86-95bd-4d1d1de901b5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:12.285209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d94ce1-c29e-4b86-95bd-4d1d1de901b5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:12.302704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b0f3a-e6d7-4e05-86d8-a7c663b1eea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:12.303401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b0f3a-e6d7-4e05-86d8-a7c663b1eea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:12.303694 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b0f3a-e6d7-4e05-86d8-a7c663b1eea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:12.303978 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b0f3a-e6d7-4e05-86d8-a7c663b1eea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:12.307683 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b0f3a-e6d7-4e05-86d8-a7c663b1eea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:12.311133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b0f3a-e6d7-4e05-86d8-a7c663b1eea1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:12.341719 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 866/1710] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:29:12.345104 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a13a2-04d2-435f-a11c-bb33fbcaf53e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:12.367829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a13a2-04d2-435f-a11c-bb33fbcaf53e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:12.375021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0f20a-b432-42ea-9f73-933be89a729c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:12.377831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0f20a-b432-42ea-9f73-933be89a729c admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:12.378521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0f20a-b432-42ea-9f73-933be89a729c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:12.379133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0f20a-b432-42ea-9f73-933be89a729c admin admin] PATH_INFO: `/v3/projects/5748c33c592c4387ba93fa70cbfa4368/users/15216a51ec954a5fb8d4cc5d261fbd59/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:12.423243 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0f20a-b432-42ea-9f73-933be89a729c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5748c33c592c4387ba93fa70cbfa4368, user_id=15216a51ec954a5fb8d4cc5d261fbd59, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:12.430593 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0f20a-b432-42ea-9f73-933be89a729c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:12.496324 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 845/1711] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:12 2022] PUT /identity/v3/projects/5748c33c592c4387ba93fa70cbfa4368/users/15216a51ec954a5fb8d4cc5d261fbd59/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:12.507061 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d93717-6e93-453e-aa7a-1a7905385940 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:12.507353 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d93717-6e93-453e-aa7a-1a7905385940 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:12.508590 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d93717-6e93-453e-aa7a-1a7905385940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:12.517026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:12.631463 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-69d93717-6e93-453e-aa7a-1a7905385940 None None] MFA Rules not processed for user `15216a51ec954a5fb8d4cc5d261fbd59`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:12.636226 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-69d93717-6e93-453e-aa7a-1a7905385940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:12.934134 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0f20a-b432-42ea-9f73-933be89a729c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:12.952223 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 867/1712] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:13.096857 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:13.136831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0f20a-b432-42ea-9f73-933be89a729c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:13.166255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df76ba-8f9a-4669-a254-b1a811b66098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:13.168254 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df76ba-8f9a-4669-a254-b1a811b66098 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:13.169005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df76ba-8f9a-4669-a254-b1a811b66098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:13.169772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df76ba-8f9a-4669-a254-b1a811b66098 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:13.171086 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df76ba-8f9a-4669-a254-b1a811b66098 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:13.173396 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df76ba-8f9a-4669-a254-b1a811b66098 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:13.277810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 846/1713] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:12 2022] POST /identity/v3/projects => generated 387 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:13.279085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d93717-6e93-453e-aa7a-1a7905385940 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:13.294915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d93717-6e93-453e-aa7a-1a7905385940 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:13.594008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54672e71-9c82-4904-8401-a8c3a57111ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:13.594835 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54672e71-9c82-4904-8401-a8c3a57111ee admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:13.595139 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54672e71-9c82-4904-8401-a8c3a57111ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:13.595501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54672e71-9c82-4904-8401-a8c3a57111ee admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:13.600758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54672e71-9c82-4904-8401-a8c3a57111ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:13.620918 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54672e71-9c82-4904-8401-a8c3a57111ee admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:13.763113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df76ba-8f9a-4669-a254-b1a811b66098 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:13.771667 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 868/1714] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:13 2022] POST /identity/v3/users => generated 294 bytes in 493 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:29:13.775981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df76ba-8f9a-4669-a254-b1a811b66098 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:13.796495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750bd5e-0535-4792-bbe4-b96db83bfae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:13.800899 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750bd5e-0535-4792-bbe4-b96db83bfae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:13.801215 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750bd5e-0535-4792-bbe4-b96db83bfae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:13.804127 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750bd5e-0535-4792-bbe4-b96db83bfae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:13.808468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750bd5e-0535-4792-bbe4-b96db83bfae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:13.816376 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750bd5e-0535-4792-bbe4-b96db83bfae5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:13.893831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 847/1715] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:13.896039 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54672e71-9c82-4904-8401-a8c3a57111ee admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:13.920652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54672e71-9c82-4904-8401-a8c3a57111ee admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:13.985636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bbf4a-109e-468b-bdc3-9f8bee820f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:13.986588 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bbf4a-109e-468b-bdc3-9f8bee820f33 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:13.987517 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bbf4a-109e-468b-bdc3-9f8bee820f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:13.991754 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bbf4a-109e-468b-bdc3-9f8bee820f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:14.006076 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909bbf4a-109e-468b-bdc3-9f8bee820f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:14.013304 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909bbf4a-109e-468b-bdc3-9f8bee820f33 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:14.053072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 869/1716] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:14.053579 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750bd5e-0535-4792-bbe4-b96db83bfae5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:14.068326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750bd5e-0535-4792-bbe4-b96db83bfae5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:14.080135 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bbf4a-109e-468b-bdc3-9f8bee820f33 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:14.080135 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-909bbf4a-109e-468b-bdc3-9f8bee820f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:14.095832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79b207-be4b-4f1c-8203-3c0c5d733b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:14.097433 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79b207-be4b-4f1c-8203-3c0c5d733b66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:14.099183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79b207-be4b-4f1c-8203-3c0c5d733b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:14.099708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79b207-be4b-4f1c-8203-3c0c5d733b66 admin admin] PATH_INFO: `/v3/projects/cb852d9de1cc4395acc6dea68b52710e/users/3c143caacb2640b7b9426c421bf45751/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:14.152244 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bbf4a-109e-468b-bdc3-9f8bee820f33 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:14.190425 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79b207-be4b-4f1c-8203-3c0c5d733b66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb852d9de1cc4395acc6dea68b52710e, user_id=3c143caacb2640b7b9426c421bf45751, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:14.198471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79b207-be4b-4f1c-8203-3c0c5d733b66 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:14.199035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8211fe-63bf-4a01-bcd0-6122db7c296f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:14.202712 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8211fe-63bf-4a01-bcd0-6122db7c296f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:14.205793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8211fe-63bf-4a01-bcd0-6122db7c296f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:14.211406 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8211fe-63bf-4a01-bcd0-6122db7c296f admin admin] PATH_INFO: `/v3/users/efcd904b529f44f1a7511d6d9009d40a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:14.223845 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8211fe-63bf-4a01-bcd0-6122db7c296f admin admin] RBAC: Authorizing `identity:delete_user(user_id=efcd904b529f44f1a7511d6d9009d40a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:14.234500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8211fe-63bf-4a01-bcd0-6122db7c296f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:14.243801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 848/1717] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:14 2022] PUT /identity/v3/projects/cb852d9de1cc4395acc6dea68b52710e/users/3c143caacb2640b7b9426c421bf45751/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:29:14.245785 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0cc2e-f475-4f38-85ce-96584003345c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:14.246089 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0cc2e-f475-4f38-85ce-96584003345c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:14.246431 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0cc2e-f475-4f38-85ce-96584003345c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:14.313156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ddf0cc2e-f475-4f38-85ce-96584003345c None None] MFA Rules not processed for user `3c143caacb2640b7b9426c421bf45751`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:14.315835 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ddf0cc2e-f475-4f38-85ce-96584003345c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:14.579307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ee8211fe-63bf-4a01-bcd0-6122db7c296f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efcd904b529f44f1a7511d6d9009d40a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:14.623119 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ee8211fe-63bf-4a01-bcd0-6122db7c296f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efcd904b529f44f1a7511d6d9009d40a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:14.624996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ee8211fe-63bf-4a01-bcd0-6122db7c296f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efcd904b529f44f1a7511d6d9009d40a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:14.626062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ee8211fe-63bf-4a01-bcd0-6122db7c296f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efcd904b529f44f1a7511d6d9009d40a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:14.629804 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:14.648763 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ee8211fe-63bf-4a01-bcd0-6122db7c296f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efcd904b529f44f1a7511d6d9009d40a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:14.669840 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 870/1718] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:14 2022] DELETE /identity/v3/users/efcd904b529f44f1a7511d6d9009d40a => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:29:14.673562 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 849/1719] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:29:14.674344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0cc2e-f475-4f38-85ce-96584003345c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:14.674484 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db64d7-c048-48f6-8c92-5e3d678acbc4 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:14.675754 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db64d7-c048-48f6-8c92-5e3d678acbc4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:14.679623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db64d7-c048-48f6-8c92-5e3d678acbc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:14.744078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-18db64d7-c048-48f6-8c92-5e3d678acbc4 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:14.745930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-18db64d7-c048-48f6-8c92-5e3d678acbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:15.060950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 871/1720] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:15.062271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efeba0d-dad4-4606-8b82-dfda51230526 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:15.064425 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efeba0d-dad4-4606-8b82-dfda51230526 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:15.065486 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efeba0d-dad4-4606-8b82-dfda51230526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:15.074570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0cc2e-f475-4f38-85ce-96584003345c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:15.088942 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56146a4a-389d-4b4b-af87-cb40e25e7651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:15.089632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56146a4a-389d-4b4b-af87-cb40e25e7651 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:15.089919 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56146a4a-389d-4b4b-af87-cb40e25e7651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:15.090201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56146a4a-389d-4b4b-af87-cb40e25e7651 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:15.090933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56146a4a-389d-4b4b-af87-cb40e25e7651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:15.093708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56146a4a-389d-4b4b-af87-cb40e25e7651 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:15.189117 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 850/1721] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:14 2022] POST /identity/v3/projects => generated 387 bytes in 515 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:15.190394 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4efeba0d-dad4-4606-8b82-dfda51230526 None None] MFA Rules not processed for user `a09029144c454b099bea86aaa5229ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:15.192895 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4efeba0d-dad4-4606-8b82-dfda51230526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:15.193446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56146a4a-389d-4b4b-af87-cb40e25e7651 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:15.197979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-56146a4a-389d-4b4b-af87-cb40e25e7651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:15.215473 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:15.392164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 872/1722] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 18 10:29:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:15.394257 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efeba0d-dad4-4606-8b82-dfda51230526 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:15.397019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4efeba0d-dad4-4606-8b82-dfda51230526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:15.539173 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56146a4a-389d-4b4b-af87-cb40e25e7651 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:15.580608 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511c134-6747-431d-a09b-2a154f5e4d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:15.603203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511c134-6747-431d-a09b-2a154f5e4d23 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:15.604328 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511c134-6747-431d-a09b-2a154f5e4d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:15.604328 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511c134-6747-431d-a09b-2a154f5e4d23 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:15.612787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1511c134-6747-431d-a09b-2a154f5e4d23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:15.629387 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1511c134-6747-431d-a09b-2a154f5e4d23 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:15.780436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 851/1723] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:15 2022] POST /identity/v3/users => generated 300 bytes in 586 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:15.781135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511c134-6747-431d-a09b-2a154f5e4d23 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:15.786682 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1511c134-6747-431d-a09b-2a154f5e4d23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:15.796923 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efeba0d-dad4-4606-8b82-dfda51230526 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:15.806035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa3e8f-7ac2-437c-ab92-71ffcfb2fe85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:15.808718 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa3e8f-7ac2-437c-ab92-71ffcfb2fe85 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:15.809670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa3e8f-7ac2-437c-ab92-71ffcfb2fe85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:15.809960 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa3e8f-7ac2-437c-ab92-71ffcfb2fe85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:15.832428 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa3e8f-7ac2-437c-ab92-71ffcfb2fe85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:15.834586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa3e8f-7ac2-437c-ab92-71ffcfb2fe85 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:15.846677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:15.875639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 873/1724] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:15.878630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa3e8f-7ac2-437c-ab92-71ffcfb2fe85 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:15.909089 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa3e8f-7ac2-437c-ab92-71ffcfb2fe85 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:15.925698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d74ada-f770-40e5-b563-7035fd93916d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:15.927434 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d74ada-f770-40e5-b563-7035fd93916d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:15.927727 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d74ada-f770-40e5-b563-7035fd93916d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:15.928007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d74ada-f770-40e5-b563-7035fd93916d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:15.931128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d74ada-f770-40e5-b563-7035fd93916d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:15.938207 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d74ada-f770-40e5-b563-7035fd93916d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:15.978545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 874/1725] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:15.982983 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d74ada-f770-40e5-b563-7035fd93916d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:16.008306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d74ada-f770-40e5-b563-7035fd93916d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:16.019330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37dde1-70fd-4acc-91f5-95122e59d58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:16.020923 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37dde1-70fd-4acc-91f5-95122e59d58e admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:16.021132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37dde1-70fd-4acc-91f5-95122e59d58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:16.021405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37dde1-70fd-4acc-91f5-95122e59d58e admin admin] PATH_INFO: `/v3/projects/5d3ea0de2e9f48feb59250d5a02c849f/users/8d8b64fa4f7e449e813a8f0fdc098741/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:16.071084 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511c134-6747-431d-a09b-2a154f5e4d23 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:16.077458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37dde1-70fd-4acc-91f5-95122e59d58e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d3ea0de2e9f48feb59250d5a02c849f, user_id=8d8b64fa4f7e449e813a8f0fdc098741, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:16.087756 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8c8a0-f0e8-4c00-9d85-b5430c9888cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6168873f2846c8af73b903bba17eb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:16.088115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37dde1-70fd-4acc-91f5-95122e59d58e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:16.092175 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8c8a0-f0e8-4c00-9d85-b5430c9888cb service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:16.092175 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8c8a0-f0e8-4c00-9d85-b5430c9888cb service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:16.092175 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8c8a0-f0e8-4c00-9d85-b5430c9888cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:16.126762 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8c8a0-f0e8-4c00-9d85-b5430c9888cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:16.131740 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8c8a0-f0e8-4c00-9d85-b5430c9888cb service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:16.136484 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 875/1726] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:15 2022] PUT /identity/v3/projects/5d3ea0de2e9f48feb59250d5a02c849f/users/8d8b64fa4f7e449e813a8f0fdc098741/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:29:16.142608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37dde1-70fd-4acc-91f5-95122e59d58e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:16.143337 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 852/1727] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 18 10:29:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:29:16.341262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37dde1-70fd-4acc-91f5-95122e59d58e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:16.359046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a0f1b-9f05-4d45-8dd6-b81cb5da9a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:16.359749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a0f1b-9f05-4d45-8dd6-b81cb5da9a88 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:16.360046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a0f1b-9f05-4d45-8dd6-b81cb5da9a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:16.360357 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a0f1b-9f05-4d45-8dd6-b81cb5da9a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:16.361228 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a0f1b-9f05-4d45-8dd6-b81cb5da9a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:16.365049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a0f1b-9f05-4d45-8dd6-b81cb5da9a88 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:16.402916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 876/1728] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:16.407235 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8c8a0-f0e8-4c00-9d85-b5430c9888cb service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:16.425919 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8c8a0-f0e8-4c00-9d85-b5430c9888cb service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:16.474656 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a35d14-e340-4a82-8d1b-6b1f973432a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:16.477407 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a35d14-e340-4a82-8d1b-6b1f973432a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:16.477714 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a35d14-e340-4a82-8d1b-6b1f973432a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:16.478025 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a35d14-e340-4a82-8d1b-6b1f973432a2 admin admin] PATH_INFO: `/v3/projects/5d3ea0de2e9f48feb59250d5a02c849f/users/8d8b64fa4f7e449e813a8f0fdc098741/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:16.494043 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a35d14-e340-4a82-8d1b-6b1f973432a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d3ea0de2e9f48feb59250d5a02c849f, user_id=8d8b64fa4f7e449e813a8f0fdc098741, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:16.501778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a35d14-e340-4a82-8d1b-6b1f973432a2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:16.531910 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 853/1729] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:16 2022] PUT /identity/v3/projects/5d3ea0de2e9f48feb59250d5a02c849f/users/8d8b64fa4f7e449e813a8f0fdc098741/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:16.543208 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815053d-d6d0-47e5-8ca0-8bdcaf394735 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:16.543928 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815053d-d6d0-47e5-8ca0-8bdcaf394735 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:16.544400 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815053d-d6d0-47e5-8ca0-8bdcaf394735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:16.651117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b815053d-d6d0-47e5-8ca0-8bdcaf394735 None None] MFA Rules not processed for user `8d8b64fa4f7e449e813a8f0fdc098741`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:16.652441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b815053d-d6d0-47e5-8ca0-8bdcaf394735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:16.903472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 877/1730] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:16.925151 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a35d14-e340-4a82-8d1b-6b1f973432a2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:16.927487 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c4a35d14-e340-4a82-8d1b-6b1f973432a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:17.092195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:17.097266 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a35d14-e340-4a82-8d1b-6b1f973432a2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:17.103058 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0813e-f420-4d9a-9a1a-5fda9c1237a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:17.105207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0813e-f420-4d9a-9a1a-5fda9c1237a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:17.105647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0813e-f420-4d9a-9a1a-5fda9c1237a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:17.106071 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0813e-f420-4d9a-9a1a-5fda9c1237a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:17.148889 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0813e-f420-4d9a-9a1a-5fda9c1237a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:17.152603 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0813e-f420-4d9a-9a1a-5fda9c1237a0 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:17.202312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 854/1731] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:17.662572 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815053d-d6d0-47e5-8ca0-8bdcaf394735 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:17.664009 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b815053d-d6d0-47e5-8ca0-8bdcaf394735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:17.773784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:17.810294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815053d-d6d0-47e5-8ca0-8bdcaf394735 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:17.818435 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f338aa-2599-4f35-ad4b-907059c00613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:17.819509 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f338aa-2599-4f35-ad4b-907059c00613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:17.819733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f338aa-2599-4f35-ad4b-907059c00613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:17.820342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f338aa-2599-4f35-ad4b-907059c00613 admin admin] PATH_INFO: `/v3/projects/926f2ef83f424b9f830078de3486361f` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:17.822192 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f338aa-2599-4f35-ad4b-907059c00613 admin admin] RBAC: Authorizing `identity:delete_project(project_id=926f2ef83f424b9f830078de3486361f)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:17.827443 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f338aa-2599-4f35-ad4b-907059c00613 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:17.931926 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-88f338aa-2599-4f35-ad4b-907059c00613 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '926f2ef83f424b9f830078de3486361f', 'request_id': 'req-88f338aa-2599-4f35-ad4b-907059c00613'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:17.942741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-88f338aa-2599-4f35-ad4b-907059c00613 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '926f2ef83f424b9f830078de3486361f', 'request_id': 'req-88f338aa-2599-4f35-ad4b-907059c00613'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:17.950958 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-88f338aa-2599-4f35-ad4b-907059c00613 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '926f2ef83f424b9f830078de3486361f', 'request_id': 'req-88f338aa-2599-4f35-ad4b-907059c00613'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:17.985027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-88f338aa-2599-4f35-ad4b-907059c00613 admin admin] The token cache is being invalidate because project 926f2ef83f424b9f830078de3486361f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:17.985617 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-88f338aa-2599-4f35-ad4b-907059c00613 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:17.986578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-88f338aa-2599-4f35-ad4b-907059c00613 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:17.989246 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 878/1732] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:17 2022] DELETE /identity/v3/projects/926f2ef83f424b9f830078de3486361f => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:17.994626 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2fc57-cfd3-4456-ad24-2ac0d007694d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:17.995485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2fc57-cfd3-4456-ad24-2ac0d007694d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:17.995998 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2fc57-cfd3-4456-ad24-2ac0d007694d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:18.027498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c1f2fc57-cfd3-4456-ad24-2ac0d007694d None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:18.030487 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c1f2fc57-cfd3-4456-ad24-2ac0d007694d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:18.131752 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 855/1733] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:18.139260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cebf6-648c-4d3e-8cc1-d1b04b833f31 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:18.139558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cebf6-648c-4d3e-8cc1-d1b04b833f31 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:18.139839 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cebf6-648c-4d3e-8cc1-d1b04b833f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:18.180655 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2fc57-cfd3-4456-ad24-2ac0d007694d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:18.192703 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c1f2fc57-cfd3-4456-ad24-2ac0d007694d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:18.198570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4b2cebf6-648c-4d3e-8cc1-d1b04b833f31 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:18.200427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4b2cebf6-648c-4d3e-8cc1-d1b04b833f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:18.221748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 879/1734] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:18.226289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2cebf6-648c-4d3e-8cc1-d1b04b833f31 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:18.237026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2cebf6-648c-4d3e-8cc1-d1b04b833f31 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:18.246174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6951fb-46a9-4af7-a0cf-361a07b94bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:18.247198 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6951fb-46a9-4af7-a0cf-361a07b94bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:18.247562 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6951fb-46a9-4af7-a0cf-361a07b94bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:18.247966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6951fb-46a9-4af7-a0cf-361a07b94bb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:18.248831 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6951fb-46a9-4af7-a0cf-361a07b94bb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:18.249213 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6951fb-46a9-4af7-a0cf-361a07b94bb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:18.260754 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6951fb-46a9-4af7-a0cf-361a07b94bb3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:18.284686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 880/1735] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:29:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:18.286877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6951fb-46a9-4af7-a0cf-361a07b94bb3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:18.304082 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6951fb-46a9-4af7-a0cf-361a07b94bb3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:18.313143 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb351898-e1c9-48a2-86db-6d6a81cd92a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:18.313880 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2fc57-cfd3-4456-ad24-2ac0d007694d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:18.314336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb351898-e1c9-48a2-86db-6d6a81cd92a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:18.314772 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb351898-e1c9-48a2-86db-6d6a81cd92a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:18.315197 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb351898-e1c9-48a2-86db-6d6a81cd92a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:18.316232 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb351898-e1c9-48a2-86db-6d6a81cd92a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:18.318547 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb351898-e1c9-48a2-86db-6d6a81cd92a5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:18.318779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96932dea-cfe4-4705-a5d0-0ba7f66cc0d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:18.319514 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96932dea-cfe4-4705-a5d0-0ba7f66cc0d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:18.320065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96932dea-cfe4-4705-a5d0-0ba7f66cc0d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:18.320268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96932dea-cfe4-4705-a5d0-0ba7f66cc0d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:18.322297 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-96932dea-cfe4-4705-a5d0-0ba7f66cc0d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:18.355116 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 881/1736] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:18 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:29:18.359645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb351898-e1c9-48a2-86db-6d6a81cd92a5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:18.422053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96932dea-cfe4-4705-a5d0-0ba7f66cc0d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:18.423535 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96932dea-cfe4-4705-a5d0-0ba7f66cc0d1 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:18.434470 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 856/1737] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:29:18.455214 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb351898-e1c9-48a2-86db-6d6a81cd92a5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:18.461613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacffad0-4adb-41b3-b7ef-d7e7f8d5839a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:18.462720 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacffad0-4adb-41b3-b7ef-d7e7f8d5839a admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:18.463187 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacffad0-4adb-41b3-b7ef-d7e7f8d5839a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:18.463853 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacffad0-4adb-41b3-b7ef-d7e7f8d5839a admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:18.465550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacffad0-4adb-41b3-b7ef-d7e7f8d5839a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:18.468548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacffad0-4adb-41b3-b7ef-d7e7f8d5839a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:18.506360 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 882/1738] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:18 2022] POST /identity/v3/users => generated 287 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:29:18.507954 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96932dea-cfe4-4705-a5d0-0ba7f66cc0d1 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:18.523541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96932dea-cfe4-4705-a5d0-0ba7f66cc0d1 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:18.528258 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445197b-d38e-439f-89a1-1e993ac41377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:18.528957 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445197b-d38e-439f-89a1-1e993ac41377 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:18.529412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445197b-d38e-439f-89a1-1e993ac41377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:18.529695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445197b-d38e-439f-89a1-1e993ac41377 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:18.530427 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445197b-d38e-439f-89a1-1e993ac41377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:18.532057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445197b-d38e-439f-89a1-1e993ac41377 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:18.543243 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 857/1739] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:18.547424 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacffad0-4adb-41b3-b7ef-d7e7f8d5839a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:18.558864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacffad0-4adb-41b3-b7ef-d7e7f8d5839a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:18.564079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c96c2e-1197-4977-bee1-f7884a2d8183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:18.564785 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c96c2e-1197-4977-bee1-f7884a2d8183 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:18.565072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c96c2e-1197-4977-bee1-f7884a2d8183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:18.565350 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c96c2e-1197-4977-bee1-f7884a2d8183 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:18.566097 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c96c2e-1197-4977-bee1-f7884a2d8183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:18.567930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c96c2e-1197-4977-bee1-f7884a2d8183 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:18.591621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 883/1740] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:29:18.593484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445197b-d38e-439f-89a1-1e993ac41377 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:18.629620 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445197b-d38e-439f-89a1-1e993ac41377 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:18.643016 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb312651-a9c9-47f7-91b7-6a6ed541f948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:18.644586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb312651-a9c9-47f7-91b7-6a6ed541f948 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:18.645430 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb312651-a9c9-47f7-91b7-6a6ed541f948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:18.645720 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb312651-a9c9-47f7-91b7-6a6ed541f948 admin admin] PATH_INFO: `/v3/projects/5919b07a88f246f9bfa38f1f49a446c5/users/3cad7549c4a546c9897fb98a143fda9c/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:18.691036 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb312651-a9c9-47f7-91b7-6a6ed541f948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5919b07a88f246f9bfa38f1f49a446c5, user_id=3cad7549c4a546c9897fb98a143fda9c, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:18.694167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb312651-a9c9-47f7-91b7-6a6ed541f948 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:18.736055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 858/1741] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:18 2022] PUT /identity/v3/projects/5919b07a88f246f9bfa38f1f49a446c5/users/3cad7549c4a546c9897fb98a143fda9c/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:29:18.747997 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed55b49-0852-463e-a2a4-7608a0cb4d99 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:18.748132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed55b49-0852-463e-a2a4-7608a0cb4d99 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:18.748388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed55b49-0852-463e-a2a4-7608a0cb4d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:18.797007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:18.820084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7ed55b49-0852-463e-a2a4-7608a0cb4d99 None None] MFA Rules not processed for user `3cad7549c4a546c9897fb98a143fda9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:18.821868 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7ed55b49-0852-463e-a2a4-7608a0cb4d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:18.919431 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f18d1d-62a3-402c-b9ed-6809286840f3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:18.919942 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f18d1d-62a3-402c-b9ed-6809286840f3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:18.920410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f18d1d-62a3-402c-b9ed-6809286840f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:18.967811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-85f18d1d-62a3-402c-b9ed-6809286840f3 None None] MFA Rules not processed for user `10d29b7934374f34923587eda9be0ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:18.975202 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-85f18d1d-62a3-402c-b9ed-6809286840f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:18.998802 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:19.034251 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 884/1742] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:19.045880 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed55b49-0852-463e-a2a4-7608a0cb4d99 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:19.049103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7ed55b49-0852-463e-a2a4-7608a0cb4d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:19.262073 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 859/1743] 10.222.0.30 () {58 vars in 1015 bytes} [Mon Jul 18 10:29:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:29:19.303530 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed55b49-0852-463e-a2a4-7608a0cb4d99 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:19.306109 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f18d1d-62a3-402c-b9ed-6809286840f3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:19.307788 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-85f18d1d-62a3-402c-b9ed-6809286840f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:19.328730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b82750-6321-47d7-97fe-d41c9ea55e62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:19.329425 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b82750-6321-47d7-97fe-d41c9ea55e62 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:19.329763 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b82750-6321-47d7-97fe-d41c9ea55e62 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:19.330053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b82750-6321-47d7-97fe-d41c9ea55e62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:19.331671 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f18d1d-62a3-402c-b9ed-6809286840f3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:19.348671 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40659280-dacc-4ae6-8268-d7a3cf75c175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:19.349647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40659280-dacc-4ae6-8268-d7a3cf75c175 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:19.350174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40659280-dacc-4ae6-8268-d7a3cf75c175 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:19.350563 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40659280-dacc-4ae6-8268-d7a3cf75c175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:19.363127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b82750-6321-47d7-97fe-d41c9ea55e62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:19.374652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40659280-dacc-4ae6-8268-d7a3cf75c175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:19.376259 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b82750-6321-47d7-97fe-d41c9ea55e62 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:19.384808 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40659280-dacc-4ae6-8268-d7a3cf75c175 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:19.398652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 860/1744] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:29:19.399809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40659280-dacc-4ae6-8268-d7a3cf75c175 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:19.401406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-40659280-dacc-4ae6-8268-d7a3cf75c175 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:19.428463 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 885/1745] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:19.434793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40659280-dacc-4ae6-8268-d7a3cf75c175 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:19.449853 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f21c7-cb2b-49d9-a636-41568dfe9430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:19.450866 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f21c7-cb2b-49d9-a636-41568dfe9430 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:19.451352 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f21c7-cb2b-49d9-a636-41568dfe9430 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:19.451931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f21c7-cb2b-49d9-a636-41568dfe9430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:19.467769 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f21c7-cb2b-49d9-a636-41568dfe9430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:19.469600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f21c7-cb2b-49d9-a636-41568dfe9430 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:19.489709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 861/1746] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:20.886664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b82750-6321-47d7-97fe-d41c9ea55e62 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:20.887941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-56b82750-6321-47d7-97fe-d41c9ea55e62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:21.070952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b82750-6321-47d7-97fe-d41c9ea55e62 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:21.094526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560045e5-d48e-49fe-90dc-669acdef9523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:21.101446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560045e5-d48e-49fe-90dc-669acdef9523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:21.101779 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560045e5-d48e-49fe-90dc-669acdef9523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:21.102014 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560045e5-d48e-49fe-90dc-669acdef9523 admin admin] PATH_INFO: `/v3/users/740613cc32554016ad9ae8f00849aa88` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:21.113431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560045e5-d48e-49fe-90dc-669acdef9523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=740613cc32554016ad9ae8f00849aa88)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:21.118891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560045e5-d48e-49fe-90dc-669acdef9523 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:21.217080 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:21.277240 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-560045e5-d48e-49fe-90dc-669acdef9523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '740613cc32554016ad9ae8f00849aa88'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:21.303893 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-560045e5-d48e-49fe-90dc-669acdef9523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '740613cc32554016ad9ae8f00849aa88'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:21.305303 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-560045e5-d48e-49fe-90dc-669acdef9523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '740613cc32554016ad9ae8f00849aa88'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:21.306214 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-560045e5-d48e-49fe-90dc-669acdef9523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '740613cc32554016ad9ae8f00849aa88'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:21.321267 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-560045e5-d48e-49fe-90dc-669acdef9523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '740613cc32554016ad9ae8f00849aa88'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:21.359902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 886/1747] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:20 2022] DELETE /identity/v3/users/740613cc32554016ad9ae8f00849aa88 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:21.361070 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376359f-8945-4ae5-a73e-2b57bae8275b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:21.361929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376359f-8945-4ae5-a73e-2b57bae8275b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:21.362361 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376359f-8945-4ae5-a73e-2b57bae8275b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:21.403464 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4376359f-8945-4ae5-a73e-2b57bae8275b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:21.405013 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4376359f-8945-4ae5-a73e-2b57bae8275b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:21.578039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 862/1748] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:21.624977 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560045e5-d48e-49fe-90dc-669acdef9523 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:21.626745 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-560045e5-d48e-49fe-90dc-669acdef9523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:21.750225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560045e5-d48e-49fe-90dc-669acdef9523 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:21.755116 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8d26a-2266-4374-bc26-268b5a0a51d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:21.756074 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8d26a-2266-4374-bc26-268b5a0a51d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:21.756433 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8d26a-2266-4374-bc26-268b5a0a51d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:21.756821 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8d26a-2266-4374-bc26-268b5a0a51d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:21.774197 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8d26a-2266-4374-bc26-268b5a0a51d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:21.779254 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8d26a-2266-4374-bc26-268b5a0a51d7 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:21.797512 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 887/1749] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:24.179097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376359f-8945-4ae5-a73e-2b57bae8275b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:24.185193 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4376359f-8945-4ae5-a73e-2b57bae8275b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:24.230382 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376359f-8945-4ae5-a73e-2b57bae8275b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:24.250192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cadd2-38f1-417b-acd0-a0b5ad57a5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:24.251557 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cadd2-38f1-417b-acd0-a0b5ad57a5ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:24.252008 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cadd2-38f1-417b-acd0-a0b5ad57a5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:24.252551 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cadd2-38f1-417b-acd0-a0b5ad57a5ac admin admin] PATH_INFO: `/v3/projects/ec5436d4818a4748894ba4c46ec362d9` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:24.255726 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cadd2-38f1-417b-acd0-a0b5ad57a5ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec5436d4818a4748894ba4c46ec362d9)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:24.263558 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cadd2-38f1-417b-acd0-a0b5ad57a5ac admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:24.322306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b38bb-cb6b-4fa0-92f9-a51dff689cf5 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:24.322868 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b38bb-cb6b-4fa0-92f9-a51dff689cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:24.322868 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b38bb-cb6b-4fa0-92f9-a51dff689cf5 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:24.324795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 888/1750] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 18 10:29:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:29:24.356025 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b382b7c-4864-4972-ab23-520cd8fcf03d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:24.356242 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b382b7c-4864-4972-ab23-520cd8fcf03d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:24.356472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b382b7c-4864-4972-ab23-520cd8fcf03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:24.369383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4d8cadd2-38f1-417b-acd0-a0b5ad57a5ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec5436d4818a4748894ba4c46ec362d9', 'request_id': 'req-4d8cadd2-38f1-417b-acd0-a0b5ad57a5ac'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:24.384847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4d8cadd2-38f1-417b-acd0-a0b5ad57a5ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec5436d4818a4748894ba4c46ec362d9', 'request_id': 'req-4d8cadd2-38f1-417b-acd0-a0b5ad57a5ac'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:24.401341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1b382b7c-4864-4972-ab23-520cd8fcf03d None None] MFA Rules not processed for user `29048c2021e340b081ae44a390583e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:24.402605 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1b382b7c-4864-4972-ab23-520cd8fcf03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:24.406323 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4d8cadd2-38f1-417b-acd0-a0b5ad57a5ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec5436d4818a4748894ba4c46ec362d9', 'request_id': 'req-4d8cadd2-38f1-417b-acd0-a0b5ad57a5ac'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:24.457002 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4d8cadd2-38f1-417b-acd0-a0b5ad57a5ac admin admin] The token cache is being invalidate because project ec5436d4818a4748894ba4c46ec362d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:24.459477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4d8cadd2-38f1-417b-acd0-a0b5ad57a5ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:24.460600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4d8cadd2-38f1-417b-acd0-a0b5ad57a5ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:24.465573 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 863/1751] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:24 2022] DELETE /identity/v3/projects/ec5436d4818a4748894ba4c46ec362d9 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:24.469772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cadd2-38f1-417b-acd0-a0b5ad57a5ac admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:24.473541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4d8cadd2-38f1-417b-acd0-a0b5ad57a5ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:24.497696 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 889/1752] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 18 10:29:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:29:24.516048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:24.639599 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b382b7c-4864-4972-ab23-520cd8fcf03d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:24.641115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1b382b7c-4864-4972-ab23-520cd8fcf03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:24.656281 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cadd2-38f1-417b-acd0-a0b5ad57a5ac admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:24.669621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164ec8d-84d0-47e1-a44d-837f0ffcfd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:24.670916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b382b7c-4864-4972-ab23-520cd8fcf03d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:24.671449 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164ec8d-84d0-47e1-a44d-837f0ffcfd6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:24.672076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164ec8d-84d0-47e1-a44d-837f0ffcfd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:24.672460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164ec8d-84d0-47e1-a44d-837f0ffcfd6f admin admin] PATH_INFO: `/v3/users/335bc08f3c544d0596a2ba7c8b77247a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:24.675802 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164ec8d-84d0-47e1-a44d-837f0ffcfd6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=335bc08f3c544d0596a2ba7c8b77247a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:24.682626 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164ec8d-84d0-47e1-a44d-837f0ffcfd6f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:24.696625 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f513ff-9013-4f47-9946-b5b13e931bb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:24.698898 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f513ff-9013-4f47-9946-b5b13e931bb0 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:24.700444 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f513ff-9013-4f47-9946-b5b13e931bb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:24.700987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f513ff-9013-4f47-9946-b5b13e931bb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:24.758195 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f513ff-9013-4f47-9946-b5b13e931bb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:24.765557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f513ff-9013-4f47-9946-b5b13e931bb0 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:24.806685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 890/1753] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:29:24.821041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6164ec8d-84d0-47e1-a44d-837f0ffcfd6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '335bc08f3c544d0596a2ba7c8b77247a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:24.821782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6164ec8d-84d0-47e1-a44d-837f0ffcfd6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '335bc08f3c544d0596a2ba7c8b77247a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:24.823621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6164ec8d-84d0-47e1-a44d-837f0ffcfd6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '335bc08f3c544d0596a2ba7c8b77247a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:24.844256 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6164ec8d-84d0-47e1-a44d-837f0ffcfd6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '335bc08f3c544d0596a2ba7c8b77247a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:24.861736 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6164ec8d-84d0-47e1-a44d-837f0ffcfd6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '335bc08f3c544d0596a2ba7c8b77247a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:24.894077 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 864/1754] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:24 2022] DELETE /identity/v3/users/335bc08f3c544d0596a2ba7c8b77247a => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:29:25.681365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f513ff-9013-4f47-9946-b5b13e931bb0 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:25.684984 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-75f513ff-9013-4f47-9946-b5b13e931bb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:25.779356 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:25.803498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08118614-7be9-408d-9f12-50a92624baf5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:25.804042 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08118614-7be9-408d-9f12-50a92624baf5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:25.804196 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08118614-7be9-408d-9f12-50a92624baf5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:25.808700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 865/1755] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 18 10:29:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:29:25.842099 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f513ff-9013-4f47-9946-b5b13e931bb0 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:25.964290 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dee1c-1f65-4d00-a822-d49f82d6a2eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:25.966014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c2c47-9190-4a3e-ae7a-3567a20d917d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:25.966187 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dee1c-1f65-4d00-a822-d49f82d6a2eb service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:25.966523 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c2c47-9190-4a3e-ae7a-3567a20d917d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:25.966909 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c2c47-9190-4a3e-ae7a-3567a20d917d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:25.967202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dee1c-1f65-4d00-a822-d49f82d6a2eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:25.968169 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dee1c-1f65-4d00-a822-d49f82d6a2eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:25.976475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d35dee1c-1f65-4d00-a822-d49f82d6a2eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:26.034847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-559c2c47-9190-4a3e-ae7a-3567a20d917d None None] MFA Rules not processed for user `29048c2021e340b081ae44a390583e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:26.039567 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-559c2c47-9190-4a3e-ae7a-3567a20d917d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:26.164240 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dee1c-1f65-4d00-a822-d49f82d6a2eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:26.165882 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dee1c-1f65-4d00-a822-d49f82d6a2eb service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:26.180910 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 891/1756] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:26.286656 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 866/1757] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 18 10:29:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:26.906209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dee1c-1f65-4d00-a822-d49f82d6a2eb service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:26.906209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d35dee1c-1f65-4d00-a822-d49f82d6a2eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:27.041548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dee1c-1f65-4d00-a822-d49f82d6a2eb service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:27.046475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1507a-4c12-4534-a2fb-8fe520957625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:27.047762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1507a-4c12-4534-a2fb-8fe520957625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:27.048145 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1507a-4c12-4534-a2fb-8fe520957625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:27.048531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1507a-4c12-4534-a2fb-8fe520957625 admin admin] PATH_INFO: `/v3/users/3cad7549c4a546c9897fb98a143fda9c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:27.049895 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1507a-4c12-4534-a2fb-8fe520957625 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cad7549c4a546c9897fb98a143fda9c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:27.052658 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1507a-4c12-4534-a2fb-8fe520957625 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:27.131994 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:27.151355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fdd1507a-4c12-4534-a2fb-8fe520957625 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cad7549c4a546c9897fb98a143fda9c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:27.188900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fdd1507a-4c12-4534-a2fb-8fe520957625 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cad7549c4a546c9897fb98a143fda9c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:27.190134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fdd1507a-4c12-4534-a2fb-8fe520957625 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cad7549c4a546c9897fb98a143fda9c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:27.190826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fdd1507a-4c12-4534-a2fb-8fe520957625 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cad7549c4a546c9897fb98a143fda9c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:27.225111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fdd1507a-4c12-4534-a2fb-8fe520957625 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cad7549c4a546c9897fb98a143fda9c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:27.241591 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c2c47-9190-4a3e-ae7a-3567a20d917d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:27.244250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-559c2c47-9190-4a3e-ae7a-3567a20d917d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:27.267020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 892/1758] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:26 2022] DELETE /identity/v3/users/3cad7549c4a546c9897fb98a143fda9c => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:27.269630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45668838-77c2-41fe-8c72-3e3dc4a99d0c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:27.269961 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45668838-77c2-41fe-8c72-3e3dc4a99d0c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:27.270358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45668838-77c2-41fe-8c72-3e3dc4a99d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:27.292588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c2c47-9190-4a3e-ae7a-3567a20d917d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:27.367443 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-45668838-77c2-41fe-8c72-3e3dc4a99d0c None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:27.371960 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-45668838-77c2-41fe-8c72-3e3dc4a99d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:27.400320 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:27.519490 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467cf029-9441-4f13-aa83-d90fa88e438e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:27.520244 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467cf029-9441-4f13-aa83-d90fa88e438e service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:27.520881 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467cf029-9441-4f13-aa83-d90fa88e438e service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:27.522432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467cf029-9441-4f13-aa83-d90fa88e438e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:27.525718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-467cf029-9441-4f13-aa83-d90fa88e438e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:27.567424 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467cf029-9441-4f13-aa83-d90fa88e438e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:27.569963 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467cf029-9441-4f13-aa83-d90fa88e438e service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:27.604443 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 867/1759] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:27.608053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467cf029-9441-4f13-aa83-d90fa88e438e service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:27.610289 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-467cf029-9441-4f13-aa83-d90fa88e438e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:27.626793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 893/1760] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:29:27.662173 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467cf029-9441-4f13-aa83-d90fa88e438e service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:27.675205 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48221ac-71f4-44e8-93b2-8948b2fdf6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:27.676992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48221ac-71f4-44e8-93b2-8948b2fdf6c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:27.680102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48221ac-71f4-44e8-93b2-8948b2fdf6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:27.680848 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48221ac-71f4-44e8-93b2-8948b2fdf6c6 admin admin] PATH_INFO: `/v3/projects/c1b45fbfe4704b8a8a0415be67da69b5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:27.684525 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48221ac-71f4-44e8-93b2-8948b2fdf6c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1b45fbfe4704b8a8a0415be67da69b5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:27.692629 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48221ac-71f4-44e8-93b2-8948b2fdf6c6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:27.784095 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45668838-77c2-41fe-8c72-3e3dc4a99d0c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:27.787435 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-45668838-77c2-41fe-8c72-3e3dc4a99d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:27.810301 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b48221ac-71f4-44e8-93b2-8948b2fdf6c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1b45fbfe4704b8a8a0415be67da69b5', 'request_id': 'req-b48221ac-71f4-44e8-93b2-8948b2fdf6c6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:27.819076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b48221ac-71f4-44e8-93b2-8948b2fdf6c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1b45fbfe4704b8a8a0415be67da69b5', 'request_id': 'req-b48221ac-71f4-44e8-93b2-8948b2fdf6c6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:27.826718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b48221ac-71f4-44e8-93b2-8948b2fdf6c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1b45fbfe4704b8a8a0415be67da69b5', 'request_id': 'req-b48221ac-71f4-44e8-93b2-8948b2fdf6c6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:27.847397 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b48221ac-71f4-44e8-93b2-8948b2fdf6c6 admin admin] The token cache is being invalidate because project c1b45fbfe4704b8a8a0415be67da69b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:27.847975 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b48221ac-71f4-44e8-93b2-8948b2fdf6c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:27.849322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b48221ac-71f4-44e8-93b2-8948b2fdf6c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:27.856628 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 868/1761] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:27 2022] DELETE /identity/v3/projects/c1b45fbfe4704b8a8a0415be67da69b5 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:27.866299 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf40a6f-3696-4789-82e7-b3461641c24d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:27.866784 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf40a6f-3696-4789-82e7-b3461641c24d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:27.868690 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf40a6f-3696-4789-82e7-b3461641c24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:27.945420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ccf40a6f-3696-4789-82e7-b3461641c24d None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:27.948267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ccf40a6f-3696-4789-82e7-b3461641c24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:27.979633 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45668838-77c2-41fe-8c72-3e3dc4a99d0c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:28.092358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b68c2-9b0a-40e1-b4cf-5bc99e97106a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:28.093037 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b68c2-9b0a-40e1-b4cf-5bc99e97106a service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:28.093386 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b68c2-9b0a-40e1-b4cf-5bc99e97106a service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:28.093672 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b68c2-9b0a-40e1-b4cf-5bc99e97106a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:28.101208 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bd9b68c2-9b0a-40e1-b4cf-5bc99e97106a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:28.162528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:28.192708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 869/1762] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:28.201400 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf1ed9-a660-4270-892b-a1ba75e2adab None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:28.202124 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf1ed9-a660-4270-892b-a1ba75e2adab None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:28.203405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf1ed9-a660-4270-892b-a1ba75e2adab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:28.248300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1fdf1ed9-a660-4270-892b-a1ba75e2adab None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:28.249610 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1fdf1ed9-a660-4270-892b-a1ba75e2adab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:28.253671 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9b68c2-9b0a-40e1-b4cf-5bc99e97106a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:28.255904 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9b68c2-9b0a-40e1-b4cf-5bc99e97106a service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:28.263905 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 870/1763] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:28.264548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf1ed9-a660-4270-892b-a1ba75e2adab None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:28.277796 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf1ed9-a660-4270-892b-a1ba75e2adab None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:28.281111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 894/1764] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:28.295702 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad150e-b923-49c7-af9e-afa2ccdffad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:28.299196 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad150e-b923-49c7-af9e-afa2ccdffad0 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:28.299852 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad150e-b923-49c7-af9e-afa2ccdffad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:28.300470 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad150e-b923-49c7-af9e-afa2ccdffad0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:28.301972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad150e-b923-49c7-af9e-afa2ccdffad0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:28.302641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad150e-b923-49c7-af9e-afa2ccdffad0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:28.306629 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad150e-b923-49c7-af9e-afa2ccdffad0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:28.342559 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 871/1765] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:28.344806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b68c2-9b0a-40e1-b4cf-5bc99e97106a service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:28.359237 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b68c2-9b0a-40e1-b4cf-5bc99e97106a service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:28.367066 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac53b44-e699-4cb9-8f1d-0fd9c86ee887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:28.368072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac53b44-e699-4cb9-8f1d-0fd9c86ee887 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:28.368392 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac53b44-e699-4cb9-8f1d-0fd9c86ee887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:28.368904 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac53b44-e699-4cb9-8f1d-0fd9c86ee887 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:28.369913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac53b44-e699-4cb9-8f1d-0fd9c86ee887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:28.373587 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac53b44-e699-4cb9-8f1d-0fd9c86ee887 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:28.499541 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 895/1766] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:28 2022] POST /identity/v3/projects => generated 369 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:28.503705 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad150e-b923-49c7-af9e-afa2ccdffad0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:28.527784 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad150e-b923-49c7-af9e-afa2ccdffad0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:28.599240 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:28.669205 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a97701-7c0b-4ca0-a22d-cbc8b1a8eef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:28.669882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a97701-7c0b-4ca0-a22d-cbc8b1a8eef1 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:28.670262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a97701-7c0b-4ca0-a22d-cbc8b1a8eef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:28.670499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a97701-7c0b-4ca0-a22d-cbc8b1a8eef1 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:28.671215 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a97701-7c0b-4ca0-a22d-cbc8b1a8eef1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:28.674698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a97701-7c0b-4ca0-a22d-cbc8b1a8eef1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:28.710336 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 872/1767] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:28 2022] POST /identity/v3/users => generated 285 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:28.711576 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac53b44-e699-4cb9-8f1d-0fd9c86ee887 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:28.727115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac53b44-e699-4cb9-8f1d-0fd9c86ee887 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:28.732198 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3d6f6-e5e3-42e7-bf96-5071be3fd4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:28.732915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3d6f6-e5e3-42e7-bf96-5071be3fd4f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:28.733242 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3d6f6-e5e3-42e7-bf96-5071be3fd4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:28.733484 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3d6f6-e5e3-42e7-bf96-5071be3fd4f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:28.734300 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3d6f6-e5e3-42e7-bf96-5071be3fd4f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:28.736180 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e3d6f6-e5e3-42e7-bf96-5071be3fd4f2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:28.757961 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 896/1768] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:28.760223 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a97701-7c0b-4ca0-a22d-cbc8b1a8eef1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:28.788521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a97701-7c0b-4ca0-a22d-cbc8b1a8eef1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:28.799478 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b73aed-3b07-4d46-8988-db75e85c043b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:28.800845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b73aed-3b07-4d46-8988-db75e85c043b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:28.801656 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b73aed-3b07-4d46-8988-db75e85c043b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:28.802631 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b73aed-3b07-4d46-8988-db75e85c043b admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:28.804652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b73aed-3b07-4d46-8988-db75e85c043b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:28.808645 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b73aed-3b07-4d46-8988-db75e85c043b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:28.831510 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 873/1769] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:29:28.834121 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3d6f6-e5e3-42e7-bf96-5071be3fd4f2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:28.855342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3d6f6-e5e3-42e7-bf96-5071be3fd4f2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:28.862818 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92145295-2527-4543-8bce-5afdbeaeb252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:28.863869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92145295-2527-4543-8bce-5afdbeaeb252 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:28.864151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92145295-2527-4543-8bce-5afdbeaeb252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:28.866912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92145295-2527-4543-8bce-5afdbeaeb252 admin admin] PATH_INFO: `/v3/projects/b79a32ee0da64f9f919de187828e396c/users/a7f9a619eb624d74a8c1c489b5cf8e8f/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:28.921697 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92145295-2527-4543-8bce-5afdbeaeb252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b79a32ee0da64f9f919de187828e396c, user_id=a7f9a619eb624d74a8c1c489b5cf8e8f, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:28.926149 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92145295-2527-4543-8bce-5afdbeaeb252 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:28.964558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 897/1770] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:28 2022] PUT /identity/v3/projects/b79a32ee0da64f9f919de187828e396c/users/a7f9a619eb624d74a8c1c489b5cf8e8f/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:28.966919 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9f052-b335-4c34-90d2-4f8e51afd7bd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:28.967215 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9f052-b335-4c34-90d2-4f8e51afd7bd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:28.967509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9f052-b335-4c34-90d2-4f8e51afd7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:29.026631 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-afb9f052-b335-4c34-90d2-4f8e51afd7bd None None] MFA Rules not processed for user `a7f9a619eb624d74a8c1c489b5cf8e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:29.027887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-afb9f052-b335-4c34-90d2-4f8e51afd7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:29.194224 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 874/1771] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:29.199559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92145295-2527-4543-8bce-5afdbeaeb252 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:29.373907 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92145295-2527-4543-8bce-5afdbeaeb252 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:29.381177 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47413d90-99b0-45ca-8d52-d959e856333a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:29.382949 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47413d90-99b0-45ca-8d52-d959e856333a admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:29.384777 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47413d90-99b0-45ca-8d52-d959e856333a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:29.385280 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47413d90-99b0-45ca-8d52-d959e856333a admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:29.389823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47413d90-99b0-45ca-8d52-d959e856333a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:29.394255 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47413d90-99b0-45ca-8d52-d959e856333a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:29.507560 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 898/1772] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:29 2022] POST /identity/v3/projects => generated 367 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:29:29.510788 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9f052-b335-4c34-90d2-4f8e51afd7bd None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:29.531069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9f052-b335-4c34-90d2-4f8e51afd7bd None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:29.658519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cab71-2450-47f8-b7ee-847bd3ab5308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:29.659697 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cab71-2450-47f8-b7ee-847bd3ab5308 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:29.660848 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cab71-2450-47f8-b7ee-847bd3ab5308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:29.661378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cab71-2450-47f8-b7ee-847bd3ab5308 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:29.662268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09cab71-2450-47f8-b7ee-847bd3ab5308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:29.676995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09cab71-2450-47f8-b7ee-847bd3ab5308 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:29.716960 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 875/1773] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:29 2022] POST /identity/v3/users => generated 290 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:29.721688 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47413d90-99b0-45ca-8d52-d959e856333a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:29.767698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47413d90-99b0-45ca-8d52-d959e856333a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:29.776746 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6794206-cbd6-416d-a652-77545e75367c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:29.778587 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6794206-cbd6-416d-a652-77545e75367c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:29.779452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6794206-cbd6-416d-a652-77545e75367c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:29.796095 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6794206-cbd6-416d-a652-77545e75367c admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:29.807718 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6794206-cbd6-416d-a652-77545e75367c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:29.810863 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6794206-cbd6-416d-a652-77545e75367c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:29.868037 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 899/1774] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:29.872886 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cab71-2450-47f8-b7ee-847bd3ab5308 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:29.889304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cab71-2450-47f8-b7ee-847bd3ab5308 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:29.931477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab1790-28ca-4629-9a8f-fe0dadb21360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:29.933174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab1790-28ca-4629-9a8f-fe0dadb21360 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:29.933735 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab1790-28ca-4629-9a8f-fe0dadb21360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:29.936355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab1790-28ca-4629-9a8f-fe0dadb21360 admin admin] PATH_INFO: `/v3/projects/0e7a4e036f6e42e18c0abd2916f82534/users/63324701a2d94bf58185a283e64a2a5e/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:29.975217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:29.982525 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab1790-28ca-4629-9a8f-fe0dadb21360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e7a4e036f6e42e18c0abd2916f82534, user_id=63324701a2d94bf58185a283e64a2a5e, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:29.987100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab1790-28ca-4629-9a8f-fe0dadb21360 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:29.996733 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6794206-cbd6-416d-a652-77545e75367c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:29.999794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e6794206-cbd6-416d-a652-77545e75367c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:30.021213 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6794206-cbd6-416d-a652-77545e75367c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:30.045041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 876/1775] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:29 2022] PUT /identity/v3/projects/0e7a4e036f6e42e18c0abd2916f82534/users/63324701a2d94bf58185a283e64a2a5e/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:30.045268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ed86d-54c4-4f2d-b344-7868e1736604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:30.045604 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ed86d-54c4-4f2d-b344-7868e1736604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:30.045892 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ed86d-54c4-4f2d-b344-7868e1736604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:30.046170 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ed86d-54c4-4f2d-b344-7868e1736604 admin admin] PATH_INFO: `/v3/users/15216a51ec954a5fb8d4cc5d261fbd59` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:30.048119 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab1790-28ca-4629-9a8f-fe0dadb21360 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:30.049232 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ed86d-54c4-4f2d-b344-7868e1736604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15216a51ec954a5fb8d4cc5d261fbd59)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:30.051997 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ed86d-54c4-4f2d-b344-7868e1736604 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:30.183559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:30.196637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab1790-28ca-4629-9a8f-fe0dadb21360 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:30.199031 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ba9ed86d-54c4-4f2d-b344-7868e1736604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15216a51ec954a5fb8d4cc5d261fbd59'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:30.206520 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664374ac-22a0-4e8e-993c-8223c02eece1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:30.209529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664374ac-22a0-4e8e-993c-8223c02eece1 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:30.209660 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664374ac-22a0-4e8e-993c-8223c02eece1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:30.210335 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664374ac-22a0-4e8e-993c-8223c02eece1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:30.211384 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664374ac-22a0-4e8e-993c-8223c02eece1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:30.216300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664374ac-22a0-4e8e-993c-8223c02eece1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:30.223563 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ba9ed86d-54c4-4f2d-b344-7868e1736604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15216a51ec954a5fb8d4cc5d261fbd59'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:30.225449 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ba9ed86d-54c4-4f2d-b344-7868e1736604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15216a51ec954a5fb8d4cc5d261fbd59'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:30.226595 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ba9ed86d-54c4-4f2d-b344-7868e1736604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15216a51ec954a5fb8d4cc5d261fbd59'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:30.249740 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 877/1776] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:30.251184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ba9ed86d-54c4-4f2d-b344-7868e1736604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15216a51ec954a5fb8d4cc5d261fbd59'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:30.253749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664374ac-22a0-4e8e-993c-8223c02eece1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:30.256025 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-664374ac-22a0-4e8e-993c-8223c02eece1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:30.269068 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664374ac-22a0-4e8e-993c-8223c02eece1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:30.269890 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 900/1777] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:29 2022] DELETE /identity/v3/users/15216a51ec954a5fb8d4cc5d261fbd59 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:30.274812 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b1db4-0d2b-4e14-a9f2-d0f5998c3556 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:30.275118 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b1db4-0d2b-4e14-a9f2-d0f5998c3556 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:30.275401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b1db4-0d2b-4e14-a9f2-d0f5998c3556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:30.307055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a46b1db4-0d2b-4e14-a9f2-d0f5998c3556 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:30.308454 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a46b1db4-0d2b-4e14-a9f2-d0f5998c3556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:30.340548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b932015-3b4d-4426-91a1-9cadf60bf889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:30.341798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b932015-3b4d-4426-91a1-9cadf60bf889 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:30.342115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b932015-3b4d-4426-91a1-9cadf60bf889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:30.342398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b932015-3b4d-4426-91a1-9cadf60bf889 admin admin] PATH_INFO: `/v3/projects/0e7a4e036f6e42e18c0abd2916f82534/users/63324701a2d94bf58185a283e64a2a5e/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:30.347396 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b932015-3b4d-4426-91a1-9cadf60bf889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e7a4e036f6e42e18c0abd2916f82534, user_id=63324701a2d94bf58185a283e64a2a5e, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:30.351614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b932015-3b4d-4426-91a1-9cadf60bf889 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:30.387333 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 878/1778] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:30 2022] PUT /identity/v3/projects/0e7a4e036f6e42e18c0abd2916f82534/users/63324701a2d94bf58185a283e64a2a5e/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:29:30.392842 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 901/1779] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:29:30.400764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a72e6e-c2ec-457c-876b-fe725dfe1680 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:30.401321 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a72e6e-c2ec-457c-876b-fe725dfe1680 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:30.401820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a72e6e-c2ec-457c-876b-fe725dfe1680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:30.424073 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b1db4-0d2b-4e14-a9f2-d0f5998c3556 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:30.426780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a46b1db4-0d2b-4e14-a9f2-d0f5998c3556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:30.459548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c3a72e6e-c2ec-457c-876b-fe725dfe1680 None None] MFA Rules not processed for user `63324701a2d94bf58185a283e64a2a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:30.461319 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c3a72e6e-c2ec-457c-876b-fe725dfe1680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:30.516405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b1db4-0d2b-4e14-a9f2-d0f5998c3556 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:30.607650 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a69ca-49e1-447d-9de7-a0063b277513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:30.609146 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a69ca-49e1-447d-9de7-a0063b277513 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:30.609459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a69ca-49e1-447d-9de7-a0063b277513 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:30.609976 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a69ca-49e1-447d-9de7-a0063b277513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:30.615247 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 879/1780] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:30 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:29:30.620961 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a72e6e-c2ec-457c-876b-fe725dfe1680 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:30.624834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c3a72e6e-c2ec-457c-876b-fe725dfe1680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:30.650666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:30.656007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:30.738994 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a69ca-49e1-447d-9de7-a0063b277513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:30.741974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a69ca-49e1-447d-9de7-a0063b277513 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:30.745696 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a72e6e-c2ec-457c-876b-fe725dfe1680 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:30.750911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76c1bc-8d25-4897-8da1-41816d90f413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:30.752059 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76c1bc-8d25-4897-8da1-41816d90f413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:30.752423 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76c1bc-8d25-4897-8da1-41816d90f413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:30.752814 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76c1bc-8d25-4897-8da1-41816d90f413 admin admin] PATH_INFO: `/v3/projects/5919b07a88f246f9bfa38f1f49a446c5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:30.754338 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76c1bc-8d25-4897-8da1-41816d90f413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5919b07a88f246f9bfa38f1f49a446c5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:30.755926 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 902/1781] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:30.756589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76c1bc-8d25-4897-8da1-41816d90f413 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:30.765643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a69ca-49e1-447d-9de7-a0063b277513 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:30.770422 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ca5a69ca-49e1-447d-9de7-a0063b277513 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:30.866714 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ef76c1bc-8d25-4897-8da1-41816d90f413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5919b07a88f246f9bfa38f1f49a446c5', 'request_id': 'req-ef76c1bc-8d25-4897-8da1-41816d90f413'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:30.883643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ef76c1bc-8d25-4897-8da1-41816d90f413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5919b07a88f246f9bfa38f1f49a446c5', 'request_id': 'req-ef76c1bc-8d25-4897-8da1-41816d90f413'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:30.893188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ef76c1bc-8d25-4897-8da1-41816d90f413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5919b07a88f246f9bfa38f1f49a446c5', 'request_id': 'req-ef76c1bc-8d25-4897-8da1-41816d90f413'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:30.899765 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a69ca-49e1-447d-9de7-a0063b277513 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:30.911416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ef76c1bc-8d25-4897-8da1-41816d90f413 admin admin] The token cache is being invalidate because project 5919b07a88f246f9bfa38f1f49a446c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:30.911799 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ef76c1bc-8d25-4897-8da1-41816d90f413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:30.912602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ef76c1bc-8d25-4897-8da1-41816d90f413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:30.914585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 880/1782] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:30 2022] DELETE /identity/v3/projects/5919b07a88f246f9bfa38f1f49a446c5 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:30.920072 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e387d-81fa-4266-8c1f-905e7068b89d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:30.920072 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e387d-81fa-4266-8c1f-905e7068b89d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:30.920287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e387d-81fa-4266-8c1f-905e7068b89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:30.942829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d83e387d-81fa-4266-8c1f-905e7068b89d None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:30.944243 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d83e387d-81fa-4266-8c1f-905e7068b89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:31.035654 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4bbd55-e85a-4bde-a7b1-a6dc7f79ecc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:31.037223 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bbd55-e85a-4bde-a7b1-a6dc7f79ecc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:31.038786 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bbd55-e85a-4bde-a7b1-a6dc7f79ecc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:31.038786 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bbd55-e85a-4bde-a7b1-a6dc7f79ecc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:31.045725 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9e4bbd55-e85a-4bde-a7b1-a6dc7f79ecc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:31.064224 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 881/1783] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:31.065260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e387d-81fa-4266-8c1f-905e7068b89d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:31.067234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d83e387d-81fa-4266-8c1f-905e7068b89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:31.081696 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e387d-81fa-4266-8c1f-905e7068b89d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:31.095942 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b310c-6357-408f-b6ac-b30e6af2fc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:31.096900 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b310c-6357-408f-b6ac-b30e6af2fc75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:31.097189 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b310c-6357-408f-b6ac-b30e6af2fc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:31.097961 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b310c-6357-408f-b6ac-b30e6af2fc75 admin admin] PATH_INFO: `/v3/users/08f5f520abda4ad29f1841af7b6c8695` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:31.099248 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b310c-6357-408f-b6ac-b30e6af2fc75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08f5f520abda4ad29f1841af7b6c8695)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:31.101837 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b310c-6357-408f-b6ac-b30e6af2fc75 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:31.172378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:31.182829 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4bbd55-e85a-4bde-a7b1-a6dc7f79ecc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:31.184772 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4bbd55-e85a-4bde-a7b1-a6dc7f79ecc6 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:31.198582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7f7b310c-6357-408f-b6ac-b30e6af2fc75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08f5f520abda4ad29f1841af7b6c8695'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:31.199751 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 903/1784] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:30 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:31.200937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e524d-2c80-49d6-a9a9-68b421601605 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:31.201048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7f7b310c-6357-408f-b6ac-b30e6af2fc75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08f5f520abda4ad29f1841af7b6c8695'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:31.201223 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e524d-2c80-49d6-a9a9-68b421601605 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:31.201503 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e524d-2c80-49d6-a9a9-68b421601605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:31.201777 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7f7b310c-6357-408f-b6ac-b30e6af2fc75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08f5f520abda4ad29f1841af7b6c8695'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:31.214208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7f7b310c-6357-408f-b6ac-b30e6af2fc75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08f5f520abda4ad29f1841af7b6c8695'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:31.229717 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7f7b310c-6357-408f-b6ac-b30e6af2fc75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08f5f520abda4ad29f1841af7b6c8695'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:31.230984 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-900e524d-2c80-49d6-a9a9-68b421601605 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:31.233126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-900e524d-2c80-49d6-a9a9-68b421601605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:31.249903 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 882/1785] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:31 2022] DELETE /identity/v3/users/08f5f520abda4ad29f1841af7b6c8695 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:31.249903 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b310c-6357-408f-b6ac-b30e6af2fc75 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:31.251458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 904/1786] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:31.251572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7f7b310c-6357-408f-b6ac-b30e6af2fc75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:31.252883 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffb3dc-5c77-4e93-b779-589f914606af None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:31.253176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffb3dc-5c77-4e93-b779-589f914606af None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:31.253454 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffb3dc-5c77-4e93-b779-589f914606af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:31.274227 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-79ffb3dc-5c77-4e93-b779-589f914606af None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:31.275750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-79ffb3dc-5c77-4e93-b779-589f914606af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:31.290670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:31.328884 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b310c-6357-408f-b6ac-b30e6af2fc75 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:31.334875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9859e18-730b-4702-adc1-918e51fb8f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:31.335738 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9859e18-730b-4702-adc1-918e51fb8f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:31.336013 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9859e18-730b-4702-adc1-918e51fb8f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:31.336289 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9859e18-730b-4702-adc1-918e51fb8f11 admin admin] PATH_INFO: `/v3/projects/5748c33c592c4387ba93fa70cbfa4368` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:31.337507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9859e18-730b-4702-adc1-918e51fb8f11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5748c33c592c4387ba93fa70cbfa4368)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:31.339423 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9859e18-730b-4702-adc1-918e51fb8f11 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:31.343927 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 905/1787] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:31.344992 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffb3dc-5c77-4e93-b779-589f914606af None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:31.354748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffb3dc-5c77-4e93-b779-589f914606af None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:31.360071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb05e69-b064-4a75-ac28-9191e372dfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:31.360850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb05e69-b064-4a75-ac28-9191e372dfdd admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:31.361142 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb05e69-b064-4a75-ac28-9191e372dfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:31.361428 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb05e69-b064-4a75-ac28-9191e372dfdd admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:31.362227 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb05e69-b064-4a75-ac28-9191e372dfdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:31.362545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb05e69-b064-4a75-ac28-9191e372dfdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:31.364734 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb05e69-b064-4a75-ac28-9191e372dfdd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:31.381085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 906/1788] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:31.381970 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb05e69-b064-4a75-ac28-9191e372dfdd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:31.385812 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8fb05e69-b064-4a75-ac28-9191e372dfdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:31.404216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e9859e18-730b-4702-adc1-918e51fb8f11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5748c33c592c4387ba93fa70cbfa4368', 'request_id': 'req-e9859e18-730b-4702-adc1-918e51fb8f11'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:31.410602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e9859e18-730b-4702-adc1-918e51fb8f11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5748c33c592c4387ba93fa70cbfa4368', 'request_id': 'req-e9859e18-730b-4702-adc1-918e51fb8f11'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:31.417492 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e9859e18-730b-4702-adc1-918e51fb8f11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5748c33c592c4387ba93fa70cbfa4368', 'request_id': 'req-e9859e18-730b-4702-adc1-918e51fb8f11'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:31.445440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e9859e18-730b-4702-adc1-918e51fb8f11 admin admin] The token cache is being invalidate because project 5748c33c592c4387ba93fa70cbfa4368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:31.445826 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e9859e18-730b-4702-adc1-918e51fb8f11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:31.446682 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e9859e18-730b-4702-adc1-918e51fb8f11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:31.451588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 883/1789] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:31 2022] DELETE /identity/v3/projects/5748c33c592c4387ba93fa70cbfa4368 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:31.451588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9859e18-730b-4702-adc1-918e51fb8f11 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:31.451588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e9859e18-730b-4702-adc1-918e51fb8f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:31.481835 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb05e69-b064-4a75-ac28-9191e372dfdd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:31.510162 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:31.526285 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9859e18-730b-4702-adc1-918e51fb8f11 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:31.529989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e7b72-c310-4f63-9cd9-b34e34d5d437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:31.530644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e7b72-c310-4f63-9cd9-b34e34d5d437 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:31.531891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e7b72-c310-4f63-9cd9-b34e34d5d437 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:31.531891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e7b72-c310-4f63-9cd9-b34e34d5d437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:31.533465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c71e7b72-c310-4f63-9cd9-b34e34d5d437 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:31.544232 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7dcc83-6c2e-43bf-8bc5-112fe0fe0a6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:31.545078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7dcc83-6c2e-43bf-8bc5-112fe0fe0a6f service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:31.545461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7dcc83-6c2e-43bf-8bc5-112fe0fe0a6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:31.545833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7dcc83-6c2e-43bf-8bc5-112fe0fe0a6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:31.547900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0c7dcc83-6c2e-43bf-8bc5-112fe0fe0a6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:31.621267 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7dcc83-6c2e-43bf-8bc5-112fe0fe0a6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:31.622413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e7b72-c310-4f63-9cd9-b34e34d5d437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:31.623421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7dcc83-6c2e-43bf-8bc5-112fe0fe0a6f service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:31.624102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e7b72-c310-4f63-9cd9-b34e34d5d437 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:31.633140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 907/1790] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:31 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:31.634115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7dcc83-6c2e-43bf-8bc5-112fe0fe0a6f service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:31.635393 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0c7dcc83-6c2e-43bf-8bc5-112fe0fe0a6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:31.641961 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 884/1791] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:31.643048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e7b72-c310-4f63-9cd9-b34e34d5d437 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:31.645034 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c71e7b72-c310-4f63-9cd9-b34e34d5d437 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:31.654037 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7dcc83-6c2e-43bf-8bc5-112fe0fe0a6f service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:31.657245 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e7b72-c310-4f63-9cd9-b34e34d5d437 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:31.661393 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba707b-fde5-436a-80d5-6d853d2b6c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:31.662580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba707b-fde5-436a-80d5-6d853d2b6c0b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:31.663247 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb457b-b785-4001-ba7a-f20da2a26aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:31.663377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba707b-fde5-436a-80d5-6d853d2b6c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:31.663776 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba707b-fde5-436a-80d5-6d853d2b6c0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:31.664312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb457b-b785-4001-ba7a-f20da2a26aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:31.664664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba707b-fde5-436a-80d5-6d853d2b6c0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:31.664776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb457b-b785-4001-ba7a-f20da2a26aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:31.665075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb457b-b785-4001-ba7a-f20da2a26aeb admin admin] PATH_INFO: `/v3/users/3c143caacb2640b7b9426c421bf45751` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:31.666362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb457b-b785-4001-ba7a-f20da2a26aeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c143caacb2640b7b9426c421bf45751)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:31.667178 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba707b-fde5-436a-80d5-6d853d2b6c0b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:31.668424 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb457b-b785-4001-ba7a-f20da2a26aeb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:31.690069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:31.710563 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 908/1792] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:31 2022] POST /identity/v3/projects => generated 405 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:31.714431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba707b-fde5-436a-80d5-6d853d2b6c0b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:31.743091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4ecb457b-b785-4001-ba7a-f20da2a26aeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c143caacb2640b7b9426c421bf45751'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:31.744486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4ecb457b-b785-4001-ba7a-f20da2a26aeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c143caacb2640b7b9426c421bf45751'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:31.745697 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4ecb457b-b785-4001-ba7a-f20da2a26aeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c143caacb2640b7b9426c421bf45751'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:31.755522 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4ecb457b-b785-4001-ba7a-f20da2a26aeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c143caacb2640b7b9426c421bf45751'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:31.771719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4ecb457b-b785-4001-ba7a-f20da2a26aeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c143caacb2640b7b9426c421bf45751'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:31.792090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 885/1793] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:31 2022] DELETE /identity/v3/users/3c143caacb2640b7b9426c421bf45751 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:31.827674 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba707b-fde5-436a-80d5-6d853d2b6c0b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:31.924648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c29569-bd6d-40e5-b126-df2b0c951977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:31.925458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c29569-bd6d-40e5-b126-df2b0c951977 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:31.925899 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c29569-bd6d-40e5-b126-df2b0c951977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:31.926314 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c29569-bd6d-40e5-b126-df2b0c951977 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:31.927100 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c29569-bd6d-40e5-b126-df2b0c951977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:31.929716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c29569-bd6d-40e5-b126-df2b0c951977 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:31.992345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 909/1794] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:31 2022] POST /identity/v3/users => generated 303 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:31.993921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb457b-b785-4001-ba7a-f20da2a26aeb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:31.995567 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4ecb457b-b785-4001-ba7a-f20da2a26aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:32.005739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c29569-bd6d-40e5-b126-df2b0c951977 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:32.011701 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-88c29569-bd6d-40e5-b126-df2b0c951977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:32.040476 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb457b-b785-4001-ba7a-f20da2a26aeb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:32.064015 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c29569-bd6d-40e5-b126-df2b0c951977 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:32.064788 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7ac94-6fde-4cbb-81f7-8f454b513108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:32.066085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7ac94-6fde-4cbb-81f7-8f454b513108 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:32.072053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7ac94-6fde-4cbb-81f7-8f454b513108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:32.073958 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7ac94-6fde-4cbb-81f7-8f454b513108 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:32.076903 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7ac94-6fde-4cbb-81f7-8f454b513108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:32.078602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7ac94-6fde-4cbb-81f7-8f454b513108 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:32.085827 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3566c7a-850a-4cb7-bebe-44acebfb6383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:32.086764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3566c7a-850a-4cb7-bebe-44acebfb6383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:32.087141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3566c7a-850a-4cb7-bebe-44acebfb6383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:32.087413 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3566c7a-850a-4cb7-bebe-44acebfb6383 admin admin] PATH_INFO: `/v3/projects/23fa6d35c6354d7e9d12cf6bf8d33556` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:32.088837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3566c7a-850a-4cb7-bebe-44acebfb6383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23fa6d35c6354d7e9d12cf6bf8d33556)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:32.091905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3566c7a-850a-4cb7-bebe-44acebfb6383 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:32.112230 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 886/1795] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:32.116461 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7ac94-6fde-4cbb-81f7-8f454b513108 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:32.127978 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7ac94-6fde-4cbb-81f7-8f454b513108 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:32.133809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985450d-169c-47a6-9d6e-d7a82c5eab2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:32.134619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985450d-169c-47a6-9d6e-d7a82c5eab2c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:32.135007 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985450d-169c-47a6-9d6e-d7a82c5eab2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:32.135367 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985450d-169c-47a6-9d6e-d7a82c5eab2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:32.136230 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985450d-169c-47a6-9d6e-d7a82c5eab2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:32.138165 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985450d-169c-47a6-9d6e-d7a82c5eab2c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:32.162078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 887/1796] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:29:32.167038 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985450d-169c-47a6-9d6e-d7a82c5eab2c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:32.184951 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985450d-169c-47a6-9d6e-d7a82c5eab2c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:32.187528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d3566c7a-850a-4cb7-bebe-44acebfb6383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23fa6d35c6354d7e9d12cf6bf8d33556', 'request_id': 'req-d3566c7a-850a-4cb7-bebe-44acebfb6383'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.194660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d3566c7a-850a-4cb7-bebe-44acebfb6383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23fa6d35c6354d7e9d12cf6bf8d33556', 'request_id': 'req-d3566c7a-850a-4cb7-bebe-44acebfb6383'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.202710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d3566c7a-850a-4cb7-bebe-44acebfb6383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23fa6d35c6354d7e9d12cf6bf8d33556', 'request_id': 'req-d3566c7a-850a-4cb7-bebe-44acebfb6383'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.216975 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d3566c7a-850a-4cb7-bebe-44acebfb6383 admin admin] The token cache is being invalidate because project 23fa6d35c6354d7e9d12cf6bf8d33556 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:32.217261 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d3566c7a-850a-4cb7-bebe-44acebfb6383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.218013 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d3566c7a-850a-4cb7-bebe-44acebfb6383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.222356 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 910/1797] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:32 2022] DELETE /identity/v3/projects/23fa6d35c6354d7e9d12cf6bf8d33556 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:32.223839 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3566c7a-850a-4cb7-bebe-44acebfb6383 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:32.225276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d3566c7a-850a-4cb7-bebe-44acebfb6383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:32.236126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:32.243719 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:32.271241 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b03ef-6d6d-428f-8b7b-5ad75e2a648d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:32.272141 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b03ef-6d6d-428f-8b7b-5ad75e2a648d admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:32.272433 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b03ef-6d6d-428f-8b7b-5ad75e2a648d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:32.272763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b03ef-6d6d-428f-8b7b-5ad75e2a648d admin admin] PATH_INFO: `/v3/projects/bc2138537f4c4975b491b3329b4662ac/users/c971fb5c6ecd4a78b0f710451aa06676/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:32.291580 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b03ef-6d6d-428f-8b7b-5ad75e2a648d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc2138537f4c4975b491b3329b4662ac, user_id=c971fb5c6ecd4a78b0f710451aa06676, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:32.295602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b03ef-6d6d-428f-8b7b-5ad75e2a648d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:32.316882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 888/1798] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:32 2022] PUT /identity/v3/projects/bc2138537f4c4975b491b3329b4662ac/users/c971fb5c6ecd4a78b0f710451aa06676/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:32.326931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320a160-4ed5-428d-9e5d-98bb12d7b5fc None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:32.327205 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3566c7a-850a-4cb7-bebe-44acebfb6383 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:32.327405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320a160-4ed5-428d-9e5d-98bb12d7b5fc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:32.327757 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320a160-4ed5-428d-9e5d-98bb12d7b5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:32.357457 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f320a160-4ed5-428d-9e5d-98bb12d7b5fc None None] MFA Rules not processed for user `c971fb5c6ecd4a78b0f710451aa06676`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:32.359052 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f320a160-4ed5-428d-9e5d-98bb12d7b5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:32.416604 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce4c1e-0b8d-4c30-89fc-3a16a0303e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:32.417659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce4c1e-0b8d-4c30-89fc-3a16a0303e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:32.418009 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce4c1e-0b8d-4c30-89fc-3a16a0303e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:32.418355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce4c1e-0b8d-4c30-89fc-3a16a0303e53 admin admin] PATH_INFO: `/v3/users/eb9f9fd743b046f387b5b7a832b917ea` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:32.419660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce4c1e-0b8d-4c30-89fc-3a16a0303e53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb9f9fd743b046f387b5b7a832b917ea)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:32.421767 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce4c1e-0b8d-4c30-89fc-3a16a0303e53 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:32.437344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 889/1799] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:32 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:29:32.438212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320a160-4ed5-428d-9e5d-98bb12d7b5fc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:32.439540 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f320a160-4ed5-428d-9e5d-98bb12d7b5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:32.453200 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320a160-4ed5-428d-9e5d-98bb12d7b5fc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:32.458555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97400d0e-623f-4a23-af02-dafe8b2210cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:32.459428 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97400d0e-623f-4a23-af02-dafe8b2210cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:32.459715 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97400d0e-623f-4a23-af02-dafe8b2210cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:32.459997 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97400d0e-623f-4a23-af02-dafe8b2210cf admin admin] PATH_INFO: `/v3/projects/cb852d9de1cc4395acc6dea68b52710e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:32.461409 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97400d0e-623f-4a23-af02-dafe8b2210cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb852d9de1cc4395acc6dea68b52710e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:32.463781 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97400d0e-623f-4a23-af02-dafe8b2210cf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:32.477479 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-38ce4c1e-0b8d-4c30-89fc-3a16a0303e53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb9f9fd743b046f387b5b7a832b917ea'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.492913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-38ce4c1e-0b8d-4c30-89fc-3a16a0303e53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb9f9fd743b046f387b5b7a832b917ea'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.493748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-38ce4c1e-0b8d-4c30-89fc-3a16a0303e53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb9f9fd743b046f387b5b7a832b917ea'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.494544 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-38ce4c1e-0b8d-4c30-89fc-3a16a0303e53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb9f9fd743b046f387b5b7a832b917ea'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.503233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-38ce4c1e-0b8d-4c30-89fc-3a16a0303e53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb9f9fd743b046f387b5b7a832b917ea'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.515840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-97400d0e-623f-4a23-af02-dafe8b2210cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb852d9de1cc4395acc6dea68b52710e', 'request_id': 'req-97400d0e-623f-4a23-af02-dafe8b2210cf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.524710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 911/1800] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:32 2022] DELETE /identity/v3/users/eb9f9fd743b046f387b5b7a832b917ea => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:29:32.524854 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-97400d0e-623f-4a23-af02-dafe8b2210cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb852d9de1cc4395acc6dea68b52710e', 'request_id': 'req-97400d0e-623f-4a23-af02-dafe8b2210cf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.524962 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce4c1e-0b8d-4c30-89fc-3a16a0303e53 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:32.526575 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-38ce4c1e-0b8d-4c30-89fc-3a16a0303e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:32.529946 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-97400d0e-623f-4a23-af02-dafe8b2210cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb852d9de1cc4395acc6dea68b52710e', 'request_id': 'req-97400d0e-623f-4a23-af02-dafe8b2210cf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.545518 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-97400d0e-623f-4a23-af02-dafe8b2210cf admin admin] The token cache is being invalidate because project cb852d9de1cc4395acc6dea68b52710e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:32.545933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-97400d0e-623f-4a23-af02-dafe8b2210cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.546594 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-97400d0e-623f-4a23-af02-dafe8b2210cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.548967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 890/1801] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:32 2022] DELETE /identity/v3/projects/cb852d9de1cc4395acc6dea68b52710e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:32.553442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97400d0e-623f-4a23-af02-dafe8b2210cf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:32.555042 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-97400d0e-623f-4a23-af02-dafe8b2210cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:32.580133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:32.607593 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:32.610973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce4c1e-0b8d-4c30-89fc-3a16a0303e53 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:32.618502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecff705-a8eb-4855-9d62-69bb37227c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:32.619688 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecff705-a8eb-4855-9d62-69bb37227c57 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:32.620052 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecff705-a8eb-4855-9d62-69bb37227c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:32.620317 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecff705-a8eb-4855-9d62-69bb37227c57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:32.621445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecff705-a8eb-4855-9d62-69bb37227c57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:32.625778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecff705-a8eb-4855-9d62-69bb37227c57 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:32.647584 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97400d0e-623f-4a23-af02-dafe8b2210cf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:32.653140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b24a71-f0e5-44a2-8619-5892fb43a2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:32.654289 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b24a71-f0e5-44a2-8619-5892fb43a2b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:32.654641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b24a71-f0e5-44a2-8619-5892fb43a2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:32.655036 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b24a71-f0e5-44a2-8619-5892fb43a2b9 admin admin] PATH_INFO: `/v3/users/8d8b64fa4f7e449e813a8f0fdc098741` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:32.656604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b24a71-f0e5-44a2-8619-5892fb43a2b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d8b64fa4f7e449e813a8f0fdc098741)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:32.658946 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b24a71-f0e5-44a2-8619-5892fb43a2b9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:32.692845 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 912/1802] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:32 2022] POST /identity/v3/projects => generated 405 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:32.697264 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecff705-a8eb-4855-9d62-69bb37227c57 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:32.700492 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8ecff705-a8eb-4855-9d62-69bb37227c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:32.778749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-85b24a71-f0e5-44a2-8619-5892fb43a2b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d8b64fa4f7e449e813a8f0fdc098741'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.779392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-85b24a71-f0e5-44a2-8619-5892fb43a2b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d8b64fa4f7e449e813a8f0fdc098741'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.780026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-85b24a71-f0e5-44a2-8619-5892fb43a2b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d8b64fa4f7e449e813a8f0fdc098741'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.798083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-85b24a71-f0e5-44a2-8619-5892fb43a2b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d8b64fa4f7e449e813a8f0fdc098741'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.831288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-85b24a71-f0e5-44a2-8619-5892fb43a2b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d8b64fa4f7e449e813a8f0fdc098741'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:32.867581 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 891/1803] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:32 2022] DELETE /identity/v3/users/8d8b64fa4f7e449e813a8f0fdc098741 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:32.876445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecff705-a8eb-4855-9d62-69bb37227c57 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:32.980536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b02c8f-d7bc-4eec-839e-fdd7b5817e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:32.983199 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b02c8f-d7bc-4eec-839e-fdd7b5817e3c admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:32.983906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b02c8f-d7bc-4eec-839e-fdd7b5817e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:32.984825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b02c8f-d7bc-4eec-839e-fdd7b5817e3c admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:32.986968 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b02c8f-d7bc-4eec-839e-fdd7b5817e3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:32.990917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b02c8f-d7bc-4eec-839e-fdd7b5817e3c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:33.029251 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 913/1804] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:32 2022] POST /identity/v3/users => generated 309 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:33.033653 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b24a71-f0e5-44a2-8619-5892fb43a2b9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:33.035519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-85b24a71-f0e5-44a2-8619-5892fb43a2b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:33.069018 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b24a71-f0e5-44a2-8619-5892fb43a2b9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:33.079737 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf39aa-75be-415c-9b62-c3d42b3df38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:33.081717 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf39aa-75be-415c-9b62-c3d42b3df38c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:33.085781 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf39aa-75be-415c-9b62-c3d42b3df38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:33.086531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf39aa-75be-415c-9b62-c3d42b3df38c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:33.091125 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf39aa-75be-415c-9b62-c3d42b3df38c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:33.093528 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf39aa-75be-415c-9b62-c3d42b3df38c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:33.110936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 892/1805] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:33.124113 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b02c8f-d7bc-4eec-839e-fdd7b5817e3c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:33.164665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b02c8f-d7bc-4eec-839e-fdd7b5817e3c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:33.202248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b3326-903b-4c02-a1e6-94508e448b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:33.204798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b3326-903b-4c02-a1e6-94508e448b0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:33.205561 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b3326-903b-4c02-a1e6-94508e448b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:33.206483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b3326-903b-4c02-a1e6-94508e448b0c admin admin] PATH_INFO: `/v3/projects/46a3521555344876af235461a5d06e7b/users/f00fabeb5c7c4f6c8a8751d9858b600b/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:33.235770 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b3326-903b-4c02-a1e6-94508e448b0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a3521555344876af235461a5d06e7b, user_id=f00fabeb5c7c4f6c8a8751d9858b600b, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:33.239652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b3326-903b-4c02-a1e6-94508e448b0c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:33.265839 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 914/1806] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:33 2022] PUT /identity/v3/projects/46a3521555344876af235461a5d06e7b/users/f00fabeb5c7c4f6c8a8751d9858b600b/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:33.269222 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf39aa-75be-415c-9b62-c3d42b3df38c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:33.280177 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf39aa-75be-415c-9b62-c3d42b3df38c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:33.349691 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11417b-2ca1-4d0d-b645-9ed4321c86d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:33.349691 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11417b-2ca1-4d0d-b645-9ed4321c86d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:33.349691 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11417b-2ca1-4d0d-b645-9ed4321c86d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:33.349691 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11417b-2ca1-4d0d-b645-9ed4321c86d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:33.350343 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11417b-2ca1-4d0d-b645-9ed4321c86d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:33.352094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11417b-2ca1-4d0d-b645-9ed4321c86d8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:33.367883 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 893/1807] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:33.373996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b3326-903b-4c02-a1e6-94508e448b0c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:33.383641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11417b-2ca1-4d0d-b645-9ed4321c86d8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:33.386982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8e11417b-2ca1-4d0d-b645-9ed4321c86d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:33.391964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b3326-903b-4c02-a1e6-94508e448b0c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:33.397792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b23a5-b54b-4d74-994f-e3f3050c054c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:33.398817 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b23a5-b54b-4d74-994f-e3f3050c054c admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:33.399054 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b23a5-b54b-4d74-994f-e3f3050c054c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:33.399336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b23a5-b54b-4d74-994f-e3f3050c054c admin admin] PATH_INFO: `/v3/projects/46a3521555344876af235461a5d06e7b/users/f00fabeb5c7c4f6c8a8751d9858b600b/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:33.401447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b23a5-b54b-4d74-994f-e3f3050c054c admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a3521555344876af235461a5d06e7b, user_id=f00fabeb5c7c4f6c8a8751d9858b600b, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:33.402843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11417b-2ca1-4d0d-b645-9ed4321c86d8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:33.404212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b23a5-b54b-4d74-994f-e3f3050c054c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:33.408037 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d1f3a-9645-40c5-9666-3e8d18a8694e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:33.409040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d1f3a-9645-40c5-9666-3e8d18a8694e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:33.409427 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d1f3a-9645-40c5-9666-3e8d18a8694e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:33.409742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d1f3a-9645-40c5-9666-3e8d18a8694e admin admin] PATH_INFO: `/v3/projects/7d7b922345004824ba28ccfdbd80b5c0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:33.414598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d1f3a-9645-40c5-9666-3e8d18a8694e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d7b922345004824ba28ccfdbd80b5c0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:33.430521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d1f3a-9645-40c5-9666-3e8d18a8694e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:33.453023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 915/1808] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:33 2022] PUT /identity/v3/projects/46a3521555344876af235461a5d06e7b/users/f00fabeb5c7c4f6c8a8751d9858b600b/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:33.453023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b23a5-b54b-4d74-994f-e3f3050c054c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:33.457223 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-127b23a5-b54b-4d74-994f-e3f3050c054c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:33.533024 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:33.546263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7d5d1f3a-9645-40c5-9666-3e8d18a8694e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d7b922345004824ba28ccfdbd80b5c0', 'request_id': 'req-7d5d1f3a-9645-40c5-9666-3e8d18a8694e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:33.553878 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7d5d1f3a-9645-40c5-9666-3e8d18a8694e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d7b922345004824ba28ccfdbd80b5c0', 'request_id': 'req-7d5d1f3a-9645-40c5-9666-3e8d18a8694e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:33.560836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:33.563566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7d5d1f3a-9645-40c5-9666-3e8d18a8694e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d7b922345004824ba28ccfdbd80b5c0', 'request_id': 'req-7d5d1f3a-9645-40c5-9666-3e8d18a8694e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:33.586713 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7d5d1f3a-9645-40c5-9666-3e8d18a8694e admin admin] The token cache is being invalidate because project 7d7b922345004824ba28ccfdbd80b5c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:33.587295 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7d5d1f3a-9645-40c5-9666-3e8d18a8694e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:33.588145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7d5d1f3a-9645-40c5-9666-3e8d18a8694e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:33.591206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 894/1809] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:33 2022] DELETE /identity/v3/projects/7d7b922345004824ba28ccfdbd80b5c0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:29:33.594356 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8c260-b609-4221-8fd5-0c4bb13eb63e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:33.594817 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8c260-b609-4221-8fd5-0c4bb13eb63e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:33.595316 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8c260-b609-4221-8fd5-0c4bb13eb63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:33.636589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-28d8c260-b609-4221-8fd5-0c4bb13eb63e None None] MFA Rules not processed for user `f00fabeb5c7c4f6c8a8751d9858b600b`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:33.638036 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-28d8c260-b609-4221-8fd5-0c4bb13eb63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:33.660454 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b23a5-b54b-4d74-994f-e3f3050c054c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:33.745293 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef6f04-7851-4c08-83c9-67f719c41824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:33.747950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef6f04-7851-4c08-83c9-67f719c41824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:33.748255 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef6f04-7851-4c08-83c9-67f719c41824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:33.748550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef6f04-7851-4c08-83c9-67f719c41824 admin admin] PATH_INFO: `/v3/projects/5d3ea0de2e9f48feb59250d5a02c849f` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:33.750173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef6f04-7851-4c08-83c9-67f719c41824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d3ea0de2e9f48feb59250d5a02c849f)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:33.752290 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef6f04-7851-4c08-83c9-67f719c41824 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:33.815665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 895/1810] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:33 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:33.815665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27852e-baaa-49d8-b99b-bbdb35533a0d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:33.815665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27852e-baaa-49d8-b99b-bbdb35533a0d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:33.815665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27852e-baaa-49d8-b99b-bbdb35533a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:33.832016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-66ef6f04-7851-4c08-83c9-67f719c41824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d3ea0de2e9f48feb59250d5a02c849f', 'request_id': 'req-66ef6f04-7851-4c08-83c9-67f719c41824'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:33.839714 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-66ef6f04-7851-4c08-83c9-67f719c41824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d3ea0de2e9f48feb59250d5a02c849f', 'request_id': 'req-66ef6f04-7851-4c08-83c9-67f719c41824'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:33.846689 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-66ef6f04-7851-4c08-83c9-67f719c41824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d3ea0de2e9f48feb59250d5a02c849f', 'request_id': 'req-66ef6f04-7851-4c08-83c9-67f719c41824'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:33.868975 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cc27852e-baaa-49d8-b99b-bbdb35533a0d None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:33.869287 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-66ef6f04-7851-4c08-83c9-67f719c41824 admin admin] The token cache is being invalidate because project 5d3ea0de2e9f48feb59250d5a02c849f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:33.870186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-66ef6f04-7851-4c08-83c9-67f719c41824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:33.870576 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cc27852e-baaa-49d8-b99b-bbdb35533a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:33.871277 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-66ef6f04-7851-4c08-83c9-67f719c41824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:33.873305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 916/1811] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:33 2022] DELETE /identity/v3/projects/5d3ea0de2e9f48feb59250d5a02c849f => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:33.880890 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef6f04-7851-4c08-83c9-67f719c41824 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:33.882285 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-66ef6f04-7851-4c08-83c9-67f719c41824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:33.950203 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef6f04-7851-4c08-83c9-67f719c41824 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:33.954796 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ab47e-4188-4e98-882f-d0022c91daa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:33.955462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ab47e-4188-4e98-882f-d0022c91daa4 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:33.955756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ab47e-4188-4e98-882f-d0022c91daa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:33.956017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ab47e-4188-4e98-882f-d0022c91daa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:33.959312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-468ab47e-4188-4e98-882f-d0022c91daa4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:33.976339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:34.083338 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 896/1812] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:34.083338 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31f7cb-7506-4f19-b69e-3a474e9a3dbe None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.083338 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31f7cb-7506-4f19-b69e-3a474e9a3dbe None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.083338 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b31f7cb-7506-4f19-b69e-3a474e9a3dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.092994 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ab47e-4188-4e98-882f-d0022c91daa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:34.094460 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ab47e-4188-4e98-882f-d0022c91daa4 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:34.112069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5b31f7cb-7506-4f19-b69e-3a474e9a3dbe None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:34.113400 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5b31f7cb-7506-4f19-b69e-3a474e9a3dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:34.113762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 917/1813] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:33 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:34.115280 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f8758-65ad-442e-a52e-4463ed323bb9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.115582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f8758-65ad-442e-a52e-4463ed323bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.115918 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f8758-65ad-442e-a52e-4463ed323bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.131912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 897/1814] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:34.135881 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e06b0-366d-461b-ae40-d016be68754c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.136381 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e06b0-366d-461b-ae40-d016be68754c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.136893 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e06b0-366d-461b-ae40-d016be68754c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.137426 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fd2f8758-65ad-442e-a52e-4463ed323bb9 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:34.139000 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fd2f8758-65ad-442e-a52e-4463ed323bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:34.149364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 918/1815] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:34.154393 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f8758-65ad-442e-a52e-4463ed323bb9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:34.161047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ca5e06b0-366d-461b-ae40-d016be68754c None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:34.162327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ca5e06b0-366d-461b-ae40-d016be68754c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:34.165336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f8758-65ad-442e-a52e-4463ed323bb9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:34.170790 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839cd93-6ffc-407b-84d9-d0f215236483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:34.172321 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839cd93-6ffc-407b-84d9-d0f215236483 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.173066 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839cd93-6ffc-407b-84d9-d0f215236483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.173653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839cd93-6ffc-407b-84d9-d0f215236483 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.174515 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839cd93-6ffc-407b-84d9-d0f215236483 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:34.174922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839cd93-6ffc-407b-84d9-d0f215236483 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:34.176834 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839cd93-6ffc-407b-84d9-d0f215236483 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:34.178843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 898/1816] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:29:34.183461 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e06b0-366d-461b-ae40-d016be68754c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:34.196109 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e06b0-366d-461b-ae40-d016be68754c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:34.201852 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057b0fa-ec02-4521-9781-9bb5dac3d0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:34.202484 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 919/1817] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:34.202681 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057b0fa-ec02-4521-9781-9bb5dac3d0da admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.203047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057b0fa-ec02-4521-9781-9bb5dac3d0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.203331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057b0fa-ec02-4521-9781-9bb5dac3d0da admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.204072 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057b0fa-ec02-4521-9781-9bb5dac3d0da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:34.204428 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057b0fa-ec02-4521-9781-9bb5dac3d0da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:34.206445 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057b0fa-ec02-4521-9781-9bb5dac3d0da admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:34.206689 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839cd93-6ffc-407b-84d9-d0f215236483 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:34.220610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839cd93-6ffc-407b-84d9-d0f215236483 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:34.227402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07a3a5-6f75-4944-be9b-100bc9f15177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:34.227971 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07a3a5-6f75-4944-be9b-100bc9f15177 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.228332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07a3a5-6f75-4944-be9b-100bc9f15177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.228740 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07a3a5-6f75-4944-be9b-100bc9f15177 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.230019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07a3a5-6f75-4944-be9b-100bc9f15177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:34.232058 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07a3a5-6f75-4944-be9b-100bc9f15177 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:34.237124 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 899/1818] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:34.242025 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057b0fa-ec02-4521-9781-9bb5dac3d0da admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:34.254453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057b0fa-ec02-4521-9781-9bb5dac3d0da admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:34.259252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06891b4f-fd22-4a4a-b046-61810a264f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:34.259997 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06891b4f-fd22-4a4a-b046-61810a264f06 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.260357 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06891b4f-fd22-4a4a-b046-61810a264f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.260739 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06891b4f-fd22-4a4a-b046-61810a264f06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.261532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06891b4f-fd22-4a4a-b046-61810a264f06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:34.263879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06891b4f-fd22-4a4a-b046-61810a264f06 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:34.276543 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 920/1819] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:34 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:34.280748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07a3a5-6f75-4944-be9b-100bc9f15177 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:34.288549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:34.299156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 900/1820] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:34 2022] POST /identity/v3/projects => generated 389 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:34.302502 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06891b4f-fd22-4a4a-b046-61810a264f06 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:34.377942 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07a3a5-6f75-4944-be9b-100bc9f15177 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:34.395781 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06891b4f-fd22-4a4a-b046-61810a264f06 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:34.400969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809be67-8484-4911-8f4c-83bb9d3abd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:34.401988 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809be67-8484-4911-8f4c-83bb9d3abd27 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.402373 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809be67-8484-4911-8f4c-83bb9d3abd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.402704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809be67-8484-4911-8f4c-83bb9d3abd27 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.403494 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809be67-8484-4911-8f4c-83bb9d3abd27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:34.405602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809be67-8484-4911-8f4c-83bb9d3abd27 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:34.433457 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 901/1821] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:34 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:34.433457 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809be67-8484-4911-8f4c-83bb9d3abd27 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:34.435477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5809be67-8484-4911-8f4c-83bb9d3abd27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:34.446534 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835e5be-560a-4045-b9ed-e09c4138904d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:34.447252 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835e5be-560a-4045-b9ed-e09c4138904d admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.447533 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835e5be-560a-4045-b9ed-e09c4138904d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.447808 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835e5be-560a-4045-b9ed-e09c4138904d admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.448551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835e5be-560a-4045-b9ed-e09c4138904d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:34.450495 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835e5be-560a-4045-b9ed-e09c4138904d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:34.453577 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809be67-8484-4911-8f4c-83bb9d3abd27 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:34.459468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bf7d4-9086-4361-857e-0e87985a92e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:34.461136 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bf7d4-9086-4361-857e-0e87985a92e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.461439 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bf7d4-9086-4361-857e-0e87985a92e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.461926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bf7d4-9086-4361-857e-0e87985a92e3 admin admin] PATH_INFO: `/v3/users/c971fb5c6ecd4a78b0f710451aa06676` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.463268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991bf7d4-9086-4361-857e-0e87985a92e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c971fb5c6ecd4a78b0f710451aa06676)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:34.465547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991bf7d4-9086-4361-857e-0e87985a92e3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:34.473371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 921/1822] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:34 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:34.474569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835e5be-560a-4045-b9ed-e09c4138904d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:34.487538 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835e5be-560a-4045-b9ed-e09c4138904d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:34.493737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0fe70-fdfa-4834-8909-3818f2eedaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:34.494563 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0fe70-fdfa-4834-8909-3818f2eedaaa admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.494955 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0fe70-fdfa-4834-8909-3818f2eedaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.495384 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0fe70-fdfa-4834-8909-3818f2eedaaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.496225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0fe70-fdfa-4834-8909-3818f2eedaaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:34.498797 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0fe70-fdfa-4834-8909-3818f2eedaaa admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:34.500658 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:34.516208 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 922/1823] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:34.517440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0fe70-fdfa-4834-8909-3818f2eedaaa admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:34.531995 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0fe70-fdfa-4834-8909-3818f2eedaaa admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:34.536320 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-991bf7d4-9086-4361-857e-0e87985a92e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c971fb5c6ecd4a78b0f710451aa06676'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:34.538119 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-991bf7d4-9086-4361-857e-0e87985a92e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c971fb5c6ecd4a78b0f710451aa06676'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:34.539538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-991bf7d4-9086-4361-857e-0e87985a92e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c971fb5c6ecd4a78b0f710451aa06676'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:34.540417 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd1b68-9e46-44a1-aa19-58ba8b9faa12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:34.541433 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd1b68-9e46-44a1-aa19-58ba8b9faa12 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.541783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd1b68-9e46-44a1-aa19-58ba8b9faa12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.542125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd1b68-9e46-44a1-aa19-58ba8b9faa12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.543372 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd1b68-9e46-44a1-aa19-58ba8b9faa12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:34.546264 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd1b68-9e46-44a1-aa19-58ba8b9faa12 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:34.547670 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-991bf7d4-9086-4361-857e-0e87985a92e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c971fb5c6ecd4a78b0f710451aa06676'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:34.562173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 923/1824] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:34.563795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd1b68-9e46-44a1-aa19-58ba8b9faa12 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:34.565873 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b6cd1b68-9e46-44a1-aa19-58ba8b9faa12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:34.569804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-991bf7d4-9086-4361-857e-0e87985a92e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c971fb5c6ecd4a78b0f710451aa06676'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:34.581757 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd1b68-9e46-44a1-aa19-58ba8b9faa12 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:34.595838 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb8566-f712-4018-942d-2889edf9321d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:34.595838 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb8566-f712-4018-942d-2889edf9321d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.595838 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb8566-f712-4018-942d-2889edf9321d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.595838 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb8566-f712-4018-942d-2889edf9321d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.596134 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 902/1825] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:34 2022] DELETE /identity/v3/users/c971fb5c6ecd4a78b0f710451aa06676 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:34.596134 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bf7d4-9086-4361-857e-0e87985a92e3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:34.596274 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb8566-f712-4018-942d-2889edf9321d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:34.596274 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb8566-f712-4018-942d-2889edf9321d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:34.596483 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-991bf7d4-9086-4361-857e-0e87985a92e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:34.615363 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 924/1826] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:34.616748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc1ed4-57f2-4ce5-909d-1bb6d135a1b9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.617051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc1ed4-57f2-4ce5-909d-1bb6d135a1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.617380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc1ed4-57f2-4ce5-909d-1bb6d135a1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.641024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-aedc1ed4-57f2-4ce5-909d-1bb6d135a1b9 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:34.642799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-aedc1ed4-57f2-4ce5-909d-1bb6d135a1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:34.684285 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bf7d4-9086-4361-857e-0e87985a92e3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:34.689361 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22de17-bd8c-477b-9c33-315e58c4d93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:34.690188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22de17-bd8c-477b-9c33-315e58c4d93d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.690299 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22de17-bd8c-477b-9c33-315e58c4d93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.690616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22de17-bd8c-477b-9c33-315e58c4d93d admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.691893 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22de17-bd8c-477b-9c33-315e58c4d93d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:34.693687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb22de17-bd8c-477b-9c33-315e58c4d93d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:34.696975 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:34.706635 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 903/1827] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:34.707923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22de17-bd8c-477b-9c33-315e58c4d93d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:34.719337 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22de17-bd8c-477b-9c33-315e58c4d93d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:34.724841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92f5ca-4865-455f-ba63-0037de663921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:34.726049 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92f5ca-4865-455f-ba63-0037de663921 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.726190 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92f5ca-4865-455f-ba63-0037de663921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.726549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92f5ca-4865-455f-ba63-0037de663921 admin admin] PATH_INFO: `/v3/projects/1862de91f75f4e969ef3c80179ab9e92/users/5202a5fe72d5415eb681ad813c14a90c/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.738861 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 925/1828] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:34.740068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc1ed4-57f2-4ce5-909d-1bb6d135a1b9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:34.746019 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92f5ca-4865-455f-ba63-0037de663921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1862de91f75f4e969ef3c80179ab9e92, user_id=5202a5fe72d5415eb681ad813c14a90c, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:34.748898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92f5ca-4865-455f-ba63-0037de663921 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:34.751627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc1ed4-57f2-4ce5-909d-1bb6d135a1b9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:34.758349 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf5e3d-5993-493e-ae54-c8e3ea1e3db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:34.759252 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf5e3d-5993-493e-ae54-c8e3ea1e3db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.759555 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf5e3d-5993-493e-ae54-c8e3ea1e3db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.760449 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf5e3d-5993-493e-ae54-c8e3ea1e3db5 admin admin] PATH_INFO: `/v3/projects/310f1a2f1f84472c9b611bbcf002da26/users/9a5173d73c9a475c998a586eca7bb913/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.776609 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 904/1829] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:34 2022] PUT /identity/v3/projects/1862de91f75f4e969ef3c80179ab9e92/users/5202a5fe72d5415eb681ad813c14a90c/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:34.776609 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92f5ca-4865-455f-ba63-0037de663921 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:34.784431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf5e3d-5993-493e-ae54-c8e3ea1e3db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=310f1a2f1f84472c9b611bbcf002da26, user_id=9a5173d73c9a475c998a586eca7bb913, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:34.785728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9f92f5ca-4865-455f-ba63-0037de663921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:34.789861 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf5e3d-5993-493e-ae54-c8e3ea1e3db5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:34.812593 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 926/1830] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:34 2022] PUT /identity/v3/projects/310f1a2f1f84472c9b611bbcf002da26/users/9a5173d73c9a475c998a586eca7bb913/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:34.814047 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4bed5-800a-4bc7-968e-fcce08530e07 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.814346 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4bed5-800a-4bc7-968e-fcce08530e07 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.814622 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4bed5-800a-4bc7-968e-fcce08530e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.854135 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2da4bed5-800a-4bc7-968e-fcce08530e07 None None] MFA Rules not processed for user `5202a5fe72d5415eb681ad813c14a90c`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:34.857138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2da4bed5-800a-4bc7-968e-fcce08530e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:34.872472 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92f5ca-4865-455f-ba63-0037de663921 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:34.951090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4a743-3a7c-446a-ba18-4cd11b23a6b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:34.951090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4a743-3a7c-446a-ba18-4cd11b23a6b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.951090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4a743-3a7c-446a-ba18-4cd11b23a6b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.951090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4a743-3a7c-446a-ba18-4cd11b23a6b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.952133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 927/1831] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:34.954090 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb575d-5c55-4c3e-abbc-dfb2990d2f69 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:34.954090 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb575d-5c55-4c3e-abbc-dfb2990d2f69 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:34.954473 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb575d-5c55-4c3e-abbc-dfb2990d2f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:34.983448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8dfb575d-5c55-4c3e-abbc-dfb2990d2f69 None None] MFA Rules not processed for user `9a5173d73c9a475c998a586eca7bb913`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:34.984775 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8dfb575d-5c55-4c3e-abbc-dfb2990d2f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:35.012941 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4a743-3a7c-446a-ba18-4cd11b23a6b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:35.014648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4a743-3a7c-446a-ba18-4cd11b23a6b9 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:35.020385 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:35.025627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 905/1832] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:35.026423 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4a743-3a7c-446a-ba18-4cd11b23a6b9 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:35.037857 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4a743-3a7c-446a-ba18-4cd11b23a6b9 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:35.054067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb628f10-e70c-41dd-bf5d-e65164fbb2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:35.054924 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb628f10-e70c-41dd-bf5d-e65164fbb2f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:35.055216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb628f10-e70c-41dd-bf5d-e65164fbb2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:35.055490 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb628f10-e70c-41dd-bf5d-e65164fbb2f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:35.056198 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb628f10-e70c-41dd-bf5d-e65164fbb2f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:35.058168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb628f10-e70c-41dd-bf5d-e65164fbb2f7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:35.077437 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 928/1833] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:35.079738 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc41920-e6f4-4978-8d5c-284a97b4868c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:35.080330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc41920-e6f4-4978-8d5c-284a97b4868c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:35.080966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc41920-e6f4-4978-8d5c-284a97b4868c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:35.083107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 929/1834] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 18 10:29:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:29:35.083966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc41920-e6f4-4978-8d5c-284a97b4868c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:35.100078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 906/1835] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:35 2022] POST /identity/v3/projects => generated 389 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:35.104487 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb628f10-e70c-41dd-bf5d-e65164fbb2f7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:35.107173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc41920-e6f4-4978-8d5c-284a97b4868c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:35.140927 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:35.219633 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb628f10-e70c-41dd-bf5d-e65164fbb2f7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:35.226190 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02653fb2-9e75-4551-8748-1ed1ab99bfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:35.226895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02653fb2-9e75-4551-8748-1ed1ab99bfca admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:35.227653 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02653fb2-9e75-4551-8748-1ed1ab99bfca admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:35.227946 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02653fb2-9e75-4551-8748-1ed1ab99bfca admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:35.229123 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02653fb2-9e75-4551-8748-1ed1ab99bfca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:35.231635 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02653fb2-9e75-4551-8748-1ed1ab99bfca admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:35.255490 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fedfea-b2a7-40c7-b3d6-8268b42ea9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:35.256485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fedfea-b2a7-40c7-b3d6-8268b42ea9e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:35.257453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fedfea-b2a7-40c7-b3d6-8268b42ea9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:35.257453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fedfea-b2a7-40c7-b3d6-8268b42ea9e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:35.259169 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fedfea-b2a7-40c7-b3d6-8268b42ea9e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:35.261603 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fedfea-b2a7-40c7-b3d6-8268b42ea9e0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:35.281916 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 907/1836] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:35 2022] POST /identity/v3/users => generated 301 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:29:35.282318 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b07dd-0238-42bc-bcab-618de9857bce None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:35.282318 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b07dd-0238-42bc-bcab-618de9857bce None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:35.282589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b07dd-0238-42bc-bcab-618de9857bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:35.318715 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 930/1837] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:35 2022] POST /identity/v3/projects => generated 377 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:35.319249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d79b07dd-0238-42bc-bcab-618de9857bce None None] MFA Rules not processed for user `29048c2021e340b081ae44a390583e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:35.319582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fedfea-b2a7-40c7-b3d6-8268b42ea9e0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:35.320060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d79b07dd-0238-42bc-bcab-618de9857bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:35.403309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fedfea-b2a7-40c7-b3d6-8268b42ea9e0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:35.409254 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937b4da-b20f-4361-ac94-65fd361ea226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:35.410056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937b4da-b20f-4361-ac94-65fd361ea226 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:35.410418 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937b4da-b20f-4361-ac94-65fd361ea226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:35.410764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937b4da-b20f-4361-ac94-65fd361ea226 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:35.411583 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937b4da-b20f-4361-ac94-65fd361ea226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:35.413866 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937b4da-b20f-4361-ac94-65fd361ea226 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:35.426808 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 908/1838] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 18 10:29:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:35.427676 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b07dd-0238-42bc-bcab-618de9857bce None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:35.428665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 931/1839] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:35.431427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937b4da-b20f-4361-ac94-65fd361ea226 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:35.446147 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937b4da-b20f-4361-ac94-65fd361ea226 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:35.447750 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b07dd-0238-42bc-bcab-618de9857bce None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:35.452972 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314f393-09e4-4ecc-a5a1-8be772f3abd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:35.454108 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314f393-09e4-4ecc-a5a1-8be772f3abd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:35.454707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314f393-09e4-4ecc-a5a1-8be772f3abd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:35.455209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314f393-09e4-4ecc-a5a1-8be772f3abd9 admin admin] PATH_INFO: `/v3/projects/1f4257ec9790450d9ca246112b589cb6/users/5dcd9006e219408e9dd1aaecfae91222/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:35.455802 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bf7b9-7a48-4882-94c9-0912582313ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:35.457385 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bf7b9-7a48-4882-94c9-0912582313ee admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:35.457874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bf7b9-7a48-4882-94c9-0912582313ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:35.458814 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bf7b9-7a48-4882-94c9-0912582313ee admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:35.459886 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1bf7b9-7a48-4882-94c9-0912582313ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:35.462192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1bf7b9-7a48-4882-94c9-0912582313ee admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:35.476299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314f393-09e4-4ecc-a5a1-8be772f3abd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4257ec9790450d9ca246112b589cb6, user_id=5dcd9006e219408e9dd1aaecfae91222, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:35.479975 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314f393-09e4-4ecc-a5a1-8be772f3abd9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:35.498025 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 909/1840] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:35 2022] POST /identity/v3/users => generated 289 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:35.503025 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 932/1841] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:35 2022] PUT /identity/v3/projects/1f4257ec9790450d9ca246112b589cb6/users/5dcd9006e219408e9dd1aaecfae91222/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:35.504427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314f393-09e4-4ecc-a5a1-8be772f3abd9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:35.511569 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bf7b9-7a48-4882-94c9-0912582313ee admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:35.516759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cc1bf7b9-7a48-4882-94c9-0912582313ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:35.546610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:35.574365 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bf7b9-7a48-4882-94c9-0912582313ee admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:35.609705 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314f393-09e4-4ecc-a5a1-8be772f3abd9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:35.617481 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c8ff6-66cc-4dd5-9294-9fa0b192bd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:35.618220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c8ff6-66cc-4dd5-9294-9fa0b192bd80 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:35.618536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c8ff6-66cc-4dd5-9294-9fa0b192bd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:35.618815 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c8ff6-66cc-4dd5-9294-9fa0b192bd80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:35.620462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c8ff6-66cc-4dd5-9294-9fa0b192bd80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:35.622057 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c8ff6-66cc-4dd5-9294-9fa0b192bd80 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:35.639379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 933/1842] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:35.639379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c8ff6-66cc-4dd5-9294-9fa0b192bd80 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:35.651452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c8ff6-66cc-4dd5-9294-9fa0b192bd80 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:35.659272 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b40f7-a47a-42b7-98ab-5e621711f8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:35.659989 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b40f7-a47a-42b7-98ab-5e621711f8cc admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:35.660419 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b40f7-a47a-42b7-98ab-5e621711f8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:35.660753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b40f7-a47a-42b7-98ab-5e621711f8cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:35.661491 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b40f7-a47a-42b7-98ab-5e621711f8cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:35.663969 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b40f7-a47a-42b7-98ab-5e621711f8cc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:35.678942 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 934/1843] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:35.680035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b40f7-a47a-42b7-98ab-5e621711f8cc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:35.695498 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b40f7-a47a-42b7-98ab-5e621711f8cc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:35.701534 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36442fb1-597a-4869-a8c3-145e61769c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:35.701923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a8785-5c09-429b-b0af-bab6a844bef5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:35.702350 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36442fb1-597a-4869-a8c3-145e61769c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:35.702649 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36442fb1-597a-4869-a8c3-145e61769c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:35.702936 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36442fb1-597a-4869-a8c3-145e61769c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:35.703675 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36442fb1-597a-4869-a8c3-145e61769c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:35.704342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a8785-5c09-429b-b0af-bab6a844bef5 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:35.704724 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a8785-5c09-429b-b0af-bab6a844bef5 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:35.705348 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36442fb1-597a-4869-a8c3-145e61769c20 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:35.705983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a8785-5c09-429b-b0af-bab6a844bef5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:35.723952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a8785-5c09-429b-b0af-bab6a844bef5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:35.725781 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a8785-5c09-429b-b0af-bab6a844bef5 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:35.726316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 935/1844] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:35.728124 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36442fb1-597a-4869-a8c3-145e61769c20 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:35.739265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36442fb1-597a-4869-a8c3-145e61769c20 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:35.748349 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 910/1845] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:35.748708 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6e856-7f91-46a5-ab43-166eb2645b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:35.749749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6e856-7f91-46a5-ab43-166eb2645b75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:35.750100 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6e856-7f91-46a5-ab43-166eb2645b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:35.750361 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a8785-5c09-429b-b0af-bab6a844bef5 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:35.750713 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6e856-7f91-46a5-ab43-166eb2645b75 admin admin] PATH_INFO: `/v3/projects/1f4257ec9790450d9ca246112b589cb6/users/5dcd9006e219408e9dd1aaecfae91222/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:35.753534 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6e856-7f91-46a5-ab43-166eb2645b75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4257ec9790450d9ca246112b589cb6, user_id=5dcd9006e219408e9dd1aaecfae91222, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:35.757090 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6e856-7f91-46a5-ab43-166eb2645b75 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:35.765273 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a8785-5c09-429b-b0af-bab6a844bef5 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:35.770839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:35.780707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 936/1846] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:35 2022] PUT /identity/v3/projects/1f4257ec9790450d9ca246112b589cb6/users/5dcd9006e219408e9dd1aaecfae91222/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:35.787043 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4670e5-5190-43f8-bdf3-acc9748e50dc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:35.788020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4670e5-5190-43f8-bdf3-acc9748e50dc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:35.790315 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4670e5-5190-43f8-bdf3-acc9748e50dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:35.836614 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0a4670e5-5190-43f8-bdf3-acc9748e50dc None None] MFA Rules not processed for user `5dcd9006e219408e9dd1aaecfae91222`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:35.838777 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0a4670e5-5190-43f8-bdf3-acc9748e50dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:35.902197 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c5ddf-e473-45ab-9aab-783f1a1ae99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:35.904198 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c5ddf-e473-45ab-9aab-783f1a1ae99b admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:35.905374 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c5ddf-e473-45ab-9aab-783f1a1ae99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:35.906009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c5ddf-e473-45ab-9aab-783f1a1ae99b admin admin] PATH_INFO: `/v3/projects/58cc630e90f745fdade9c4740eba7d65/users/b7c5612789904891b1a318a0fca232ae/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:35.933438 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c5ddf-e473-45ab-9aab-783f1a1ae99b admin admin] RBAC: Authorizing `identity:create_grant(project_id=58cc630e90f745fdade9c4740eba7d65, user_id=b7c5612789904891b1a318a0fca232ae, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:35.936968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c5ddf-e473-45ab-9aab-783f1a1ae99b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:35.962323 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 911/1847] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:35 2022] PUT /identity/v3/projects/58cc630e90f745fdade9c4740eba7d65/users/b7c5612789904891b1a318a0fca232ae/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:35.963754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c5ddf-e473-45ab-9aab-783f1a1ae99b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:35.976022 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-964c5ddf-e473-45ab-9aab-783f1a1ae99b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:36.024763 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 937/1848] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:36.068533 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7492ef-1322-45a9-a941-cd7394c0edc0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:36.068533 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7492ef-1322-45a9-a941-cd7394c0edc0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:36.068533 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7492ef-1322-45a9-a941-cd7394c0edc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:36.113428 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c5ddf-e473-45ab-9aab-783f1a1ae99b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:36.123022 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c283fc-be5f-4d2c-a4e9-8877c6e24d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:36.124100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c283fc-be5f-4d2c-a4e9-8877c6e24d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:36.124558 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c283fc-be5f-4d2c-a4e9-8877c6e24d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:36.124979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c283fc-be5f-4d2c-a4e9-8877c6e24d6b admin admin] PATH_INFO: `/v3/projects/bc2138537f4c4975b491b3329b4662ac` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:36.127808 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c283fc-be5f-4d2c-a4e9-8877c6e24d6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc2138537f4c4975b491b3329b4662ac)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:36.129951 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c283fc-be5f-4d2c-a4e9-8877c6e24d6b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:36.135589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8e7492ef-1322-45a9-a941-cd7394c0edc0 None None] MFA Rules not processed for user `b7c5612789904891b1a318a0fca232ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:36.136977 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8e7492ef-1322-45a9-a941-cd7394c0edc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:36.198551 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b1c283fc-be5f-4d2c-a4e9-8877c6e24d6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc2138537f4c4975b491b3329b4662ac', 'request_id': 'req-b1c283fc-be5f-4d2c-a4e9-8877c6e24d6b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:36.206030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b1c283fc-be5f-4d2c-a4e9-8877c6e24d6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc2138537f4c4975b491b3329b4662ac', 'request_id': 'req-b1c283fc-be5f-4d2c-a4e9-8877c6e24d6b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:36.207968 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 938/1849] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:36.212253 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7492ef-1322-45a9-a941-cd7394c0edc0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:36.213688 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8e7492ef-1322-45a9-a941-cd7394c0edc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:36.215138 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b1c283fc-be5f-4d2c-a4e9-8877c6e24d6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc2138537f4c4975b491b3329b4662ac', 'request_id': 'req-b1c283fc-be5f-4d2c-a4e9-8877c6e24d6b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:36.218966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:36.234426 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b1c283fc-be5f-4d2c-a4e9-8877c6e24d6b admin admin] The token cache is being invalidate because project bc2138537f4c4975b491b3329b4662ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:36.235141 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b1c283fc-be5f-4d2c-a4e9-8877c6e24d6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:36.235685 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b1c283fc-be5f-4d2c-a4e9-8877c6e24d6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:36.238107 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 912/1850] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:35 2022] DELETE /identity/v3/projects/bc2138537f4c4975b491b3329b4662ac => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:36.238957 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c283fc-be5f-4d2c-a4e9-8877c6e24d6b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:36.240335 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b1c283fc-be5f-4d2c-a4e9-8877c6e24d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:36.271126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:36.292679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7492ef-1322-45a9-a941-cd7394c0edc0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:36.321792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c283fc-be5f-4d2c-a4e9-8877c6e24d6b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:36.327078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81952328-6c54-41eb-b0ef-63aa27c0ce70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:36.328248 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81952328-6c54-41eb-b0ef-63aa27c0ce70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:36.328639 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81952328-6c54-41eb-b0ef-63aa27c0ce70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:36.329015 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81952328-6c54-41eb-b0ef-63aa27c0ce70 admin admin] PATH_INFO: `/v3/users/a7f9a619eb624d74a8c1c489b5cf8e8f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:36.330388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81952328-6c54-41eb-b0ef-63aa27c0ce70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7f9a619eb624d74a8c1c489b5cf8e8f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:36.333028 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81952328-6c54-41eb-b0ef-63aa27c0ce70 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:36.355147 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f37c7f-aa92-495a-824a-f4df8301645c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:36.356010 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f37c7f-aa92-495a-824a-f4df8301645c service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:36.356227 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f37c7f-aa92-495a-824a-f4df8301645c service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:36.356531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f37c7f-aa92-495a-824a-f4df8301645c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:36.358341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a0f37c7f-aa92-495a-824a-f4df8301645c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:36.390636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-81952328-6c54-41eb-b0ef-63aa27c0ce70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7f9a619eb624d74a8c1c489b5cf8e8f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:36.391320 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-81952328-6c54-41eb-b0ef-63aa27c0ce70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7f9a619eb624d74a8c1c489b5cf8e8f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:36.391943 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-81952328-6c54-41eb-b0ef-63aa27c0ce70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7f9a619eb624d74a8c1c489b5cf8e8f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:36.407623 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-81952328-6c54-41eb-b0ef-63aa27c0ce70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7f9a619eb624d74a8c1c489b5cf8e8f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:36.425308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-81952328-6c54-41eb-b0ef-63aa27c0ce70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7f9a619eb624d74a8c1c489b5cf8e8f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:36.438993 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f37c7f-aa92-495a-824a-f4df8301645c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:36.440478 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f37c7f-aa92-495a-824a-f4df8301645c service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:36.443432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 913/1851] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:36 2022] DELETE /identity/v3/users/a7f9a619eb624d74a8c1c489b5cf8e8f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:36.444157 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81952328-6c54-41eb-b0ef-63aa27c0ce70 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:36.445794 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-81952328-6c54-41eb-b0ef-63aa27c0ce70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:36.455831 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 939/1852] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:36.458452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f37c7f-aa92-495a-824a-f4df8301645c service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:36.461831 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a0f37c7f-aa92-495a-824a-f4df8301645c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:36.482153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f37c7f-aa92-495a-824a-f4df8301645c service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:36.557297 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81952328-6c54-41eb-b0ef-63aa27c0ce70 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:36.565140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ab789-07ce-4687-a597-33bbf7091f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:36.565993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ab789-07ce-4687-a597-33bbf7091f37 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:36.566921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ab789-07ce-4687-a597-33bbf7091f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:36.567297 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ab789-07ce-4687-a597-33bbf7091f37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:36.570227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ab789-07ce-4687-a597-33bbf7091f37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:36.574161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ab789-07ce-4687-a597-33bbf7091f37 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:36.639589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 914/1853] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:36 2022] POST /identity/v3/projects => generated 377 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:36.639589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfa9bf-931b-46f0-9a76-b4afd89fc4ea None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:36.639589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfa9bf-931b-46f0-9a76-b4afd89fc4ea None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:36.639589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfa9bf-931b-46f0-9a76-b4afd89fc4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:36.640971 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55abb07d-8ccb-4e18-85d7-712e1a38997b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:36.642284 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abb07d-8ccb-4e18-85d7-712e1a38997b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:36.642719 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abb07d-8ccb-4e18-85d7-712e1a38997b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:36.643141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abb07d-8ccb-4e18-85d7-712e1a38997b admin admin] PATH_INFO: `/v3/users/f00fabeb5c7c4f6c8a8751d9858b600b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:36.645124 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55abb07d-8ccb-4e18-85d7-712e1a38997b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f00fabeb5c7c4f6c8a8751d9858b600b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:36.647269 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55abb07d-8ccb-4e18-85d7-712e1a38997b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:36.659997 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:36.663170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6cdfa9bf-931b-46f0-9a76-b4afd89fc4ea None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:36.664644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6cdfa9bf-931b-46f0-9a76-b4afd89fc4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:36.688241 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:36.708467 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-55abb07d-8ccb-4e18-85d7-712e1a38997b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f00fabeb5c7c4f6c8a8751d9858b600b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:36.724744 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-55abb07d-8ccb-4e18-85d7-712e1a38997b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f00fabeb5c7c4f6c8a8751d9858b600b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:36.725530 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-55abb07d-8ccb-4e18-85d7-712e1a38997b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f00fabeb5c7c4f6c8a8751d9858b600b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:36.726252 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-55abb07d-8ccb-4e18-85d7-712e1a38997b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f00fabeb5c7c4f6c8a8751d9858b600b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:36.736331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-55abb07d-8ccb-4e18-85d7-712e1a38997b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f00fabeb5c7c4f6c8a8751d9858b600b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:36.740635 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 915/1854] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:36.740635 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfa9bf-931b-46f0-9a76-b4afd89fc4ea None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:36.743747 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6cdfa9bf-931b-46f0-9a76-b4afd89fc4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:36.757103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfa9bf-931b-46f0-9a76-b4afd89fc4ea None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:36.758778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 940/1855] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:36 2022] DELETE /identity/v3/users/f00fabeb5c7c4f6c8a8751d9858b600b => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:36.759852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55abb07d-8ccb-4e18-85d7-712e1a38997b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:36.761376 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-55abb07d-8ccb-4e18-85d7-712e1a38997b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:36.835434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe147ee-de76-4de7-9337-4868090d2e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:36.836202 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe147ee-de76-4de7-9337-4868090d2e4e admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:36.836592 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe147ee-de76-4de7-9337-4868090d2e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:36.836947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe147ee-de76-4de7-9337-4868090d2e4e admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:36.838082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe147ee-de76-4de7-9337-4868090d2e4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:36.840623 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe147ee-de76-4de7-9337-4868090d2e4e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:36.877222 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 916/1856] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:36 2022] POST /identity/v3/users => generated 295 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:36.879580 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe147ee-de76-4de7-9337-4868090d2e4e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:36.882077 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dfe147ee-de76-4de7-9337-4868090d2e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:36.922707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55abb07d-8ccb-4e18-85d7-712e1a38997b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:36.936871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0aa95e-6e98-4144-87be-f7b2f08c3098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:36.938032 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0aa95e-6e98-4144-87be-f7b2f08c3098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:36.938294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0aa95e-6e98-4144-87be-f7b2f08c3098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:36.938586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0aa95e-6e98-4144-87be-f7b2f08c3098 admin admin] PATH_INFO: `/v3/users/5202a5fe72d5415eb681ad813c14a90c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:36.944096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0aa95e-6e98-4144-87be-f7b2f08c3098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5202a5fe72d5415eb681ad813c14a90c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:36.946233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0aa95e-6e98-4144-87be-f7b2f08c3098 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:36.980693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe147ee-de76-4de7-9337-4868090d2e4e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:36.993503 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c591a-36b2-49e3-a34e-53d3114781c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:36.994299 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c591a-36b2-49e3-a34e-53d3114781c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:36.994591 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c591a-36b2-49e3-a34e-53d3114781c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:36.994876 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c591a-36b2-49e3-a34e-53d3114781c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:37.010553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c591a-36b2-49e3-a34e-53d3114781c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:37.014678 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c591a-36b2-49e3-a34e-53d3114781c6 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:37.033452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1c0aa95e-6e98-4144-87be-f7b2f08c3098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5202a5fe72d5415eb681ad813c14a90c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:37.039953 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 917/1857] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:37.043200 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c591a-36b2-49e3-a34e-53d3114781c6 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:37.055251 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1c0aa95e-6e98-4144-87be-f7b2f08c3098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5202a5fe72d5415eb681ad813c14a90c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:37.056381 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1c0aa95e-6e98-4144-87be-f7b2f08c3098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5202a5fe72d5415eb681ad813c14a90c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:37.057597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1c0aa95e-6e98-4144-87be-f7b2f08c3098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5202a5fe72d5415eb681ad813c14a90c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:37.063462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c591a-36b2-49e3-a34e-53d3114781c6 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:37.077244 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad2b67-79f9-405a-83e2-45de6485991d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:37.077983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad2b67-79f9-405a-83e2-45de6485991d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:37.078392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad2b67-79f9-405a-83e2-45de6485991d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:37.078604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad2b67-79f9-405a-83e2-45de6485991d admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:37.079354 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad2b67-79f9-405a-83e2-45de6485991d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:37.085567 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad2b67-79f9-405a-83e2-45de6485991d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:37.086460 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1c0aa95e-6e98-4144-87be-f7b2f08c3098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5202a5fe72d5415eb681ad813c14a90c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:37.111203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 918/1859] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:37.111396 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 941/1859] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:36 2022] DELETE /identity/v3/users/5202a5fe72d5415eb681ad813c14a90c => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:37.111518 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e718231-252f-40ef-8272-45eeea1f46e8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:37.111518 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e718231-252f-40ef-8272-45eeea1f46e8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:37.111731 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e718231-252f-40ef-8272-45eeea1f46e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:37.112759 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0aa95e-6e98-4144-87be-f7b2f08c3098 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:37.117096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1c0aa95e-6e98-4144-87be-f7b2f08c3098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:37.129132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:37.161504 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7e718231-252f-40ef-8272-45eeea1f46e8 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:37.165183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7e718231-252f-40ef-8272-45eeea1f46e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:37.291202 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:37.298038 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0aa95e-6e98-4144-87be-f7b2f08c3098 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:37.305015 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a09a91-6dbd-4691-a526-a1524375bd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:37.306810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a09a91-6dbd-4691-a526-a1524375bd1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:37.307265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a09a91-6dbd-4691-a526-a1524375bd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:37.308617 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a09a91-6dbd-4691-a526-a1524375bd1d admin admin] PATH_INFO: `/v3/projects/fb6d650c5046405b99561e4ed0a8b4a1/users/2d30ef9771354fa6b3c70100207b83a4/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:37.320591 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 919/1860] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:37.330332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a09a91-6dbd-4691-a526-a1524375bd1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb6d650c5046405b99561e4ed0a8b4a1, user_id=2d30ef9771354fa6b3c70100207b83a4, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:37.333821 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e718231-252f-40ef-8272-45eeea1f46e8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:37.334236 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a09a91-6dbd-4691-a526-a1524375bd1d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:37.336557 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7e718231-252f-40ef-8272-45eeea1f46e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:37.352753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 942/1861] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:37 2022] PUT /identity/v3/projects/fb6d650c5046405b99561e4ed0a8b4a1/users/2d30ef9771354fa6b3c70100207b83a4/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:37.356215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a09a91-6dbd-4691-a526-a1524375bd1d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:37.451307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a09a91-6dbd-4691-a526-a1524375bd1d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:37.456610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2456ba7-67c0-4716-b6c1-5b43cc056bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:37.459451 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2456ba7-67c0-4716-b6c1-5b43cc056bab admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:37.459842 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2456ba7-67c0-4716-b6c1-5b43cc056bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:37.460221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2456ba7-67c0-4716-b6c1-5b43cc056bab admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:37.461102 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2456ba7-67c0-4716-b6c1-5b43cc056bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:37.463557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2456ba7-67c0-4716-b6c1-5b43cc056bab admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:37.471169 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e718231-252f-40ef-8272-45eeea1f46e8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:37.479568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 943/1862] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:37.482915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2456ba7-67c0-4716-b6c1-5b43cc056bab admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:37.499143 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2456ba7-67c0-4716-b6c1-5b43cc056bab admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:37.507067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efc32f-8eb5-4db0-baca-2eaa5ca74e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:37.518347 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efc32f-8eb5-4db0-baca-2eaa5ca74e12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:37.521398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efc32f-8eb5-4db0-baca-2eaa5ca74e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:37.521885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efc32f-8eb5-4db0-baca-2eaa5ca74e12 admin admin] PATH_INFO: `/v3/projects/fb6d650c5046405b99561e4ed0a8b4a1/users/2d30ef9771354fa6b3c70100207b83a4/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:37.528249 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47efc32f-8eb5-4db0-baca-2eaa5ca74e12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb6d650c5046405b99561e4ed0a8b4a1, user_id=2d30ef9771354fa6b3c70100207b83a4, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:37.538306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47efc32f-8eb5-4db0-baca-2eaa5ca74e12 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:37.580283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 944/1863] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:37 2022] PUT /identity/v3/projects/fb6d650c5046405b99561e4ed0a8b4a1/users/2d30ef9771354fa6b3c70100207b83a4/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:37.580986 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a3109-8a75-4990-8fc8-ea52cd4f98c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:37.582472 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a3109-8a75-4990-8fc8-ea52cd4f98c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:37.582472 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a3109-8a75-4990-8fc8-ea52cd4f98c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:37.582785 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a3109-8a75-4990-8fc8-ea52cd4f98c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:37.598302 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89c3ba-6453-4422-b8b3-0a598d0c8135 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:37.603627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89c3ba-6453-4422-b8b3-0a598d0c8135 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:37.605190 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a3109-8a75-4990-8fc8-ea52cd4f98c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:37.605646 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89c3ba-6453-4422-b8b3-0a598d0c8135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:37.606876 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a3109-8a75-4990-8fc8-ea52cd4f98c8 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:37.629251 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 920/1864] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:29:37.634951 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a3109-8a75-4990-8fc8-ea52cd4f98c8 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:37.636638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-900a3109-8a75-4990-8fc8-ea52cd4f98c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:37.652923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a3109-8a75-4990-8fc8-ea52cd4f98c8 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:37.690581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-cf89c3ba-6453-4422-b8b3-0a598d0c8135 None None] MFA Rules not processed for user `2d30ef9771354fa6b3c70100207b83a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:37.692016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cf89c3ba-6453-4422-b8b3-0a598d0c8135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:37.790447 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725ba01-3ae0-4124-ad75-4c06716b4041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:37.791418 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725ba01-3ae0-4124-ad75-4c06716b4041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:37.791802 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725ba01-3ae0-4124-ad75-4c06716b4041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:37.792197 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725ba01-3ae0-4124-ad75-4c06716b4041 admin admin] PATH_INFO: `/v3/projects/46a3521555344876af235461a5d06e7b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:37.793786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725ba01-3ae0-4124-ad75-4c06716b4041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46a3521555344876af235461a5d06e7b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:37.797627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725ba01-3ae0-4124-ad75-4c06716b4041 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:37.832993 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:37.848668 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 945/1865] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:37.850524 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89c3ba-6453-4422-b8b3-0a598d0c8135 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:37.853037 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cf89c3ba-6453-4422-b8b3-0a598d0c8135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:37.874907 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89c3ba-6453-4422-b8b3-0a598d0c8135 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:37.897992 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9c37b-689b-48d5-a0d6-95e2097736a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:37.900811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9c37b-689b-48d5-a0d6-95e2097736a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:37.901570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9c37b-689b-48d5-a0d6-95e2097736a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:37.902424 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9c37b-689b-48d5-a0d6-95e2097736a0 admin admin] PATH_INFO: `/v3/projects/b79a32ee0da64f9f919de187828e396c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:37.904024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9c37b-689b-48d5-a0d6-95e2097736a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b79a32ee0da64f9f919de187828e396c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:37.906697 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9c37b-689b-48d5-a0d6-95e2097736a0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:37.935358 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:37.935358 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4725ba01-3ae0-4124-ad75-4c06716b4041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46a3521555344876af235461a5d06e7b', 'request_id': 'req-4725ba01-3ae0-4124-ad75-4c06716b4041'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:37.950725 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4725ba01-3ae0-4124-ad75-4c06716b4041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46a3521555344876af235461a5d06e7b', 'request_id': 'req-4725ba01-3ae0-4124-ad75-4c06716b4041'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:37.965318 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4725ba01-3ae0-4124-ad75-4c06716b4041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46a3521555344876af235461a5d06e7b', 'request_id': 'req-4725ba01-3ae0-4124-ad75-4c06716b4041'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:37.993855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4725ba01-3ae0-4124-ad75-4c06716b4041 admin admin] The token cache is being invalidate because project 46a3521555344876af235461a5d06e7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:37.994405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4725ba01-3ae0-4124-ad75-4c06716b4041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:37.994835 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4725ba01-3ae0-4124-ad75-4c06716b4041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.002936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 921/1866] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:37 2022] DELETE /identity/v3/projects/46a3521555344876af235461a5d06e7b => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:38.004268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725ba01-3ae0-4124-ad75-4c06716b4041 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:38.010009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4725ba01-3ae0-4124-ad75-4c06716b4041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:38.025793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-76b9c37b-689b-48d5-a0d6-95e2097736a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b79a32ee0da64f9f919de187828e396c', 'request_id': 'req-76b9c37b-689b-48d5-a0d6-95e2097736a0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.035477 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-76b9c37b-689b-48d5-a0d6-95e2097736a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b79a32ee0da64f9f919de187828e396c', 'request_id': 'req-76b9c37b-689b-48d5-a0d6-95e2097736a0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.047532 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-76b9c37b-689b-48d5-a0d6-95e2097736a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b79a32ee0da64f9f919de187828e396c', 'request_id': 'req-76b9c37b-689b-48d5-a0d6-95e2097736a0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.066312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-76b9c37b-689b-48d5-a0d6-95e2097736a0 admin admin] The token cache is being invalidate because project b79a32ee0da64f9f919de187828e396c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:38.066602 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-76b9c37b-689b-48d5-a0d6-95e2097736a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.069247 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-76b9c37b-689b-48d5-a0d6-95e2097736a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.072492 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 946/1867] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:37 2022] DELETE /identity/v3/projects/b79a32ee0da64f9f919de187828e396c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:38.074594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44414bf-5092-430b-9ba5-4a7d43f9e87f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:38.075661 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44414bf-5092-430b-9ba5-4a7d43f9e87f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:38.076134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44414bf-5092-430b-9ba5-4a7d43f9e87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:38.090663 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725ba01-3ae0-4124-ad75-4c06716b4041 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:38.099642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f44414bf-5092-430b-9ba5-4a7d43f9e87f None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:38.101898 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f44414bf-5092-430b-9ba5-4a7d43f9e87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:38.162021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0544b-a07b-4633-a6c4-2fa6437807f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:38.166316 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0544b-a07b-4633-a6c4-2fa6437807f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:38.166641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0544b-a07b-4633-a6c4-2fa6437807f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:38.166921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0544b-a07b-4633-a6c4-2fa6437807f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:38.169199 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-52d0544b-a07b-4633-a6c4-2fa6437807f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:38.193886 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 947/1868] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:38.200227 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44414bf-5092-430b-9ba5-4a7d43f9e87f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:38.206026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f44414bf-5092-430b-9ba5-4a7d43f9e87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:38.224149 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44414bf-5092-430b-9ba5-4a7d43f9e87f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:38.233244 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81865cb6-c012-4658-8bf7-99f45db3537c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:38.234510 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81865cb6-c012-4658-8bf7-99f45db3537c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:38.234806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81865cb6-c012-4658-8bf7-99f45db3537c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:38.235248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81865cb6-c012-4658-8bf7-99f45db3537c admin admin] PATH_INFO: `/v3/users/63324701a2d94bf58185a283e64a2a5e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:38.236886 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81865cb6-c012-4658-8bf7-99f45db3537c admin admin] RBAC: Authorizing `identity:delete_user(user_id=63324701a2d94bf58185a283e64a2a5e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:38.240115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81865cb6-c012-4658-8bf7-99f45db3537c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:38.255944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0544b-a07b-4633-a6c4-2fa6437807f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:38.257422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0544b-a07b-4633-a6c4-2fa6437807f3 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:38.268149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 922/1869] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:38 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:38.269984 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0544b-a07b-4633-a6c4-2fa6437807f3 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:38.271279 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-52d0544b-a07b-4633-a6c4-2fa6437807f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:38.287461 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0544b-a07b-4633-a6c4-2fa6437807f3 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:38.294051 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8380e-d7e0-4065-ab9c-d0d48f0c9c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:38.295145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8380e-d7e0-4065-ab9c-d0d48f0c9c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:38.295468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8380e-d7e0-4065-ab9c-d0d48f0c9c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:38.295848 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8380e-d7e0-4065-ab9c-d0d48f0c9c11 admin admin] PATH_INFO: `/v3/projects/1862de91f75f4e969ef3c80179ab9e92` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:38.297265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba8380e-d7e0-4065-ab9c-d0d48f0c9c11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1862de91f75f4e969ef3c80179ab9e92)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:38.299615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba8380e-d7e0-4065-ab9c-d0d48f0c9c11 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:38.311564 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-81865cb6-c012-4658-8bf7-99f45db3537c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63324701a2d94bf58185a283e64a2a5e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.320914 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-81865cb6-c012-4658-8bf7-99f45db3537c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63324701a2d94bf58185a283e64a2a5e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.321662 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-81865cb6-c012-4658-8bf7-99f45db3537c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63324701a2d94bf58185a283e64a2a5e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.322391 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-81865cb6-c012-4658-8bf7-99f45db3537c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63324701a2d94bf58185a283e64a2a5e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.330795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-81865cb6-c012-4658-8bf7-99f45db3537c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63324701a2d94bf58185a283e64a2a5e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.336699 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:38.350021 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 948/1870] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:38 2022] DELETE /identity/v3/users/63324701a2d94bf58185a283e64a2a5e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:38.351149 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d974b-bb6c-4644-a2e3-aac022f2c525 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:38.352081 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d974b-bb6c-4644-a2e3-aac022f2c525 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:38.356932 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d974b-bb6c-4644-a2e3-aac022f2c525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:38.358981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:38.373501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3ba8380e-d7e0-4065-ab9c-d0d48f0c9c11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1862de91f75f4e969ef3c80179ab9e92', 'request_id': 'req-3ba8380e-d7e0-4065-ab9c-d0d48f0c9c11'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.386887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3ba8380e-d7e0-4065-ab9c-d0d48f0c9c11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1862de91f75f4e969ef3c80179ab9e92', 'request_id': 'req-3ba8380e-d7e0-4065-ab9c-d0d48f0c9c11'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.395477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3ba8380e-d7e0-4065-ab9c-d0d48f0c9c11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1862de91f75f4e969ef3c80179ab9e92', 'request_id': 'req-3ba8380e-d7e0-4065-ab9c-d0d48f0c9c11'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.396361 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5a5d974b-bb6c-4644-a2e3-aac022f2c525 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:38.399447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5a5d974b-bb6c-4644-a2e3-aac022f2c525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:38.417126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3ba8380e-d7e0-4065-ab9c-d0d48f0c9c11 admin admin] The token cache is being invalidate because project 1862de91f75f4e969ef3c80179ab9e92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:38.417708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3ba8380e-d7e0-4065-ab9c-d0d48f0c9c11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.419245 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3ba8380e-d7e0-4065-ab9c-d0d48f0c9c11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.422331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 923/1871] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:38 2022] DELETE /identity/v3/projects/1862de91f75f4e969ef3c80179ab9e92 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:38.423902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8380e-d7e0-4065-ab9c-d0d48f0c9c11 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:38.426141 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3ba8380e-d7e0-4065-ab9c-d0d48f0c9c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:38.482302 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 949/1872] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:38.492740 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d974b-bb6c-4644-a2e3-aac022f2c525 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:38.494556 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5a5d974b-bb6c-4644-a2e3-aac022f2c525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:38.510283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d974b-bb6c-4644-a2e3-aac022f2c525 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:38.511358 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8380e-d7e0-4065-ab9c-d0d48f0c9c11 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:38.515802 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71607c22-7454-4f6c-be94-9605d4bffa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:38.516130 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb7b7b-32f7-418e-a8c0-78f8a08c51a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:38.516725 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71607c22-7454-4f6c-be94-9605d4bffa0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:38.516865 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb7b7b-32f7-418e-a8c0-78f8a08c51a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:38.517097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb7b7b-32f7-418e-a8c0-78f8a08c51a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:38.517212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71607c22-7454-4f6c-be94-9605d4bffa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:38.517470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71607c22-7454-4f6c-be94-9605d4bffa0b admin admin] PATH_INFO: `/v3/users/5dcd9006e219408e9dd1aaecfae91222` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:38.518325 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb7b7b-32f7-418e-a8c0-78f8a08c51a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:38.518979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71607c22-7454-4f6c-be94-9605d4bffa0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dcd9006e219408e9dd1aaecfae91222)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:38.520829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-90bb7b7b-32f7-418e-a8c0-78f8a08c51a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:38.521584 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71607c22-7454-4f6c-be94-9605d4bffa0b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:38.588006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-71607c22-7454-4f6c-be94-9605d4bffa0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dcd9006e219408e9dd1aaecfae91222'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.604620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-71607c22-7454-4f6c-be94-9605d4bffa0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dcd9006e219408e9dd1aaecfae91222'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.605421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-71607c22-7454-4f6c-be94-9605d4bffa0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dcd9006e219408e9dd1aaecfae91222'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.607487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-71607c22-7454-4f6c-be94-9605d4bffa0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dcd9006e219408e9dd1aaecfae91222'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.618715 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-71607c22-7454-4f6c-be94-9605d4bffa0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dcd9006e219408e9dd1aaecfae91222'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.622025 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb7b7b-32f7-418e-a8c0-78f8a08c51a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:38.623490 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb7b7b-32f7-418e-a8c0-78f8a08c51a3 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:38.648736 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 950/1874] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:38 2022] DELETE /identity/v3/users/5dcd9006e219408e9dd1aaecfae91222 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:38.648736 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71607c22-7454-4f6c-be94-9605d4bffa0b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:38.649229 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 924/1874] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:38.649329 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-71607c22-7454-4f6c-be94-9605d4bffa0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:38.786362 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71607c22-7454-4f6c-be94-9605d4bffa0b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:38.791388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb7b7b-32f7-418e-a8c0-78f8a08c51a3 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:38.793734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-90bb7b7b-32f7-418e-a8c0-78f8a08c51a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:38.794827 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e5e1e-a667-4e4f-a956-a2970ca772ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:38.795593 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e5e1e-a667-4e4f-a956-a2970ca772ab admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:38.795960 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e5e1e-a667-4e4f-a956-a2970ca772ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:38.796316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e5e1e-a667-4e4f-a956-a2970ca772ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:38.797193 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e5e1e-a667-4e4f-a956-a2970ca772ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:38.797637 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e5e1e-a667-4e4f-a956-a2970ca772ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:38.800115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e5e1e-a667-4e4f-a956-a2970ca772ab admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:38.821367 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb7b7b-32f7-418e-a8c0-78f8a08c51a3 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:38.823102 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:38.826886 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 951/1875] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:38.828616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3eb2fb-b4db-4513-8436-0b1423d59f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:38.829503 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3eb2fb-b4db-4513-8436-0b1423d59f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:38.829791 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3eb2fb-b4db-4513-8436-0b1423d59f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:38.830370 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3eb2fb-b4db-4513-8436-0b1423d59f76 admin admin] PATH_INFO: `/v3/projects/0e7a4e036f6e42e18c0abd2916f82534` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:38.833264 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3eb2fb-b4db-4513-8436-0b1423d59f76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e7a4e036f6e42e18c0abd2916f82534)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:38.835499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3eb2fb-b4db-4513-8436-0b1423d59f76 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:38.837539 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e5e1e-a667-4e4f-a956-a2970ca772ab admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:38.853606 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e5e1e-a667-4e4f-a956-a2970ca772ab admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:38.865583 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6c8c3-647c-4daf-9d48-efdf04d4716d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:38.866345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6c8c3-647c-4daf-9d48-efdf04d4716d admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:38.866766 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6c8c3-647c-4daf-9d48-efdf04d4716d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:38.867365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6c8c3-647c-4daf-9d48-efdf04d4716d admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:38.869037 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6c8c3-647c-4daf-9d48-efdf04d4716d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:38.873323 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6c8c3-647c-4daf-9d48-efdf04d4716d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:38.914257 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cc3eb2fb-b4db-4513-8436-0b1423d59f76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e7a4e036f6e42e18c0abd2916f82534', 'request_id': 'req-cc3eb2fb-b4db-4513-8436-0b1423d59f76'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.922553 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 952/1876] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:38 2022] POST /identity/v3/projects => generated 387 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:38.925234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cc3eb2fb-b4db-4513-8436-0b1423d59f76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e7a4e036f6e42e18c0abd2916f82534', 'request_id': 'req-cc3eb2fb-b4db-4513-8436-0b1423d59f76'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.927724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6c8c3-647c-4daf-9d48-efdf04d4716d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:38.948742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cc3eb2fb-b4db-4513-8436-0b1423d59f76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e7a4e036f6e42e18c0abd2916f82534', 'request_id': 'req-cc3eb2fb-b4db-4513-8436-0b1423d59f76'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.977166 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cc3eb2fb-b4db-4513-8436-0b1423d59f76 admin admin] The token cache is being invalidate because project 0e7a4e036f6e42e18c0abd2916f82534 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:38.977474 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cc3eb2fb-b4db-4513-8436-0b1423d59f76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.978505 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cc3eb2fb-b4db-4513-8436-0b1423d59f76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:38.981029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 925/1877] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:38 2022] DELETE /identity/v3/projects/0e7a4e036f6e42e18c0abd2916f82534 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:38.985870 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53788fc6-d2d6-437a-bf8d-67e4eaf1cb32 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:38.986178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53788fc6-d2d6-437a-bf8d-67e4eaf1cb32 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:38.986462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53788fc6-d2d6-437a-bf8d-67e4eaf1cb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:38.999055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:39.019065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-53788fc6-d2d6-437a-bf8d-67e4eaf1cb32 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:39.020866 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-53788fc6-d2d6-437a-bf8d-67e4eaf1cb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:39.084062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6c8c3-647c-4daf-9d48-efdf04d4716d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:39.125299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099b70e-7f1a-4e2a-aac5-bd296bafe5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:39.125299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099b70e-7f1a-4e2a-aac5-bd296bafe5de admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:39.125299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099b70e-7f1a-4e2a-aac5-bd296bafe5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:39.125299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099b70e-7f1a-4e2a-aac5-bd296bafe5de admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:39.125655 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099b70e-7f1a-4e2a-aac5-bd296bafe5de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:39.128234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099b70e-7f1a-4e2a-aac5-bd296bafe5de admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:39.179279 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 926/1878] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:29:39.180973 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53788fc6-d2d6-437a-bf8d-67e4eaf1cb32 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:39.198052 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 953/1879] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:38 2022] POST /identity/v3/users => generated 294 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:39.200549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-53788fc6-d2d6-437a-bf8d-67e4eaf1cb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:39.202056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49462fb0-5478-418b-89b2-89acabfa8036 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:39.202056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49462fb0-5478-418b-89b2-89acabfa8036 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:39.204160 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49462fb0-5478-418b-89b2-89acabfa8036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:39.236699 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53788fc6-d2d6-437a-bf8d-67e4eaf1cb32 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:39.255720 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c0862-ef0a-4d19-b0b5-6e817defdcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:39.256963 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c0862-ef0a-4d19-b0b5-6e817defdcf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:39.257341 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c0862-ef0a-4d19-b0b5-6e817defdcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:39.257722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c0862-ef0a-4d19-b0b5-6e817defdcf5 admin admin] PATH_INFO: `/v3/projects/1f4257ec9790450d9ca246112b589cb6` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:39.259159 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c0862-ef0a-4d19-b0b5-6e817defdcf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f4257ec9790450d9ca246112b589cb6)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:39.261293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c0862-ef0a-4d19-b0b5-6e817defdcf5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:39.270381 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-49462fb0-5478-418b-89b2-89acabfa8036 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:39.271859 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-49462fb0-5478-418b-89b2-89acabfa8036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:39.291580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 954/1880] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:39.292445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49462fb0-5478-418b-89b2-89acabfa8036 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:39.294792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-49462fb0-5478-418b-89b2-89acabfa8036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:39.327280 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49462fb0-5478-418b-89b2-89acabfa8036 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:39.328332 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-733c0862-ef0a-4d19-b0b5-6e817defdcf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f4257ec9790450d9ca246112b589cb6', 'request_id': 'req-733c0862-ef0a-4d19-b0b5-6e817defdcf5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:39.339279 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-733c0862-ef0a-4d19-b0b5-6e817defdcf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f4257ec9790450d9ca246112b589cb6', 'request_id': 'req-733c0862-ef0a-4d19-b0b5-6e817defdcf5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:39.348140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-733c0862-ef0a-4d19-b0b5-6e817defdcf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f4257ec9790450d9ca246112b589cb6', 'request_id': 'req-733c0862-ef0a-4d19-b0b5-6e817defdcf5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:39.381450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-733c0862-ef0a-4d19-b0b5-6e817defdcf5 admin admin] The token cache is being invalidate because project 1f4257ec9790450d9ca246112b589cb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:39.381747 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-733c0862-ef0a-4d19-b0b5-6e817defdcf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:39.382414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-733c0862-ef0a-4d19-b0b5-6e817defdcf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:39.384669 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 927/1881] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:39 2022] DELETE /identity/v3/projects/1f4257ec9790450d9ca246112b589cb6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:39.390017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c0862-ef0a-4d19-b0b5-6e817defdcf5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:39.392070 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-733c0862-ef0a-4d19-b0b5-6e817defdcf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:39.406882 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1649df-1cf3-4169-a616-c037862c3ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:39.407769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1649df-1cf3-4169-a616-c037862c3ecb admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:39.408130 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1649df-1cf3-4169-a616-c037862c3ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:39.408480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1649df-1cf3-4169-a616-c037862c3ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:39.409326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1649df-1cf3-4169-a616-c037862c3ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:39.411703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1649df-1cf3-4169-a616-c037862c3ecb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:39.427047 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 955/1882] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:39.427706 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f84b3-1054-480a-8bd0-baf804cbe3fa None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:39.428473 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f84b3-1054-480a-8bd0-baf804cbe3fa None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:39.428994 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f84b3-1054-480a-8bd0-baf804cbe3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:39.458285 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-922f84b3-1054-480a-8bd0-baf804cbe3fa None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:39.460483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-922f84b3-1054-480a-8bd0-baf804cbe3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:39.480669 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 956/1883] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:39.481975 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f84b3-1054-480a-8bd0-baf804cbe3fa None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:39.484230 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-922f84b3-1054-480a-8bd0-baf804cbe3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:39.498795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f84b3-1054-480a-8bd0-baf804cbe3fa None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:39.499036 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c0862-ef0a-4d19-b0b5-6e817defdcf5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:39.504629 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee573a-5dc8-4f1a-abf9-bb5c40b627ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:39.505228 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd9097-6125-42da-baf1-d2ad009d9682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:39.505399 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee573a-5dc8-4f1a-abf9-bb5c40b627ac admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:39.505587 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee573a-5dc8-4f1a-abf9-bb5c40b627ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:39.505857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee573a-5dc8-4f1a-abf9-bb5c40b627ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:39.506130 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd9097-6125-42da-baf1-d2ad009d9682 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:39.506522 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd9097-6125-42da-baf1-d2ad009d9682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:39.506642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee573a-5dc8-4f1a-abf9-bb5c40b627ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:39.506910 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd9097-6125-42da-baf1-d2ad009d9682 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:39.507740 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd9097-6125-42da-baf1-d2ad009d9682 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:39.508129 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd9097-6125-42da-baf1-d2ad009d9682 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:39.508232 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee573a-5dc8-4f1a-abf9-bb5c40b627ac admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:39.510087 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd9097-6125-42da-baf1-d2ad009d9682 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:39.520236 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 957/1884] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:39.520903 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc9fb2-cfd8-4aed-90c0-f215c5c44709 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:39.521133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc9fb2-cfd8-4aed-90c0-f215c5c44709 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:39.521415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc9fb2-cfd8-4aed-90c0-f215c5c44709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:39.531674 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 928/1885] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:39.531674 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd9097-6125-42da-baf1-d2ad009d9682 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:39.541923 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f4dc9fb2-cfd8-4aed-90c0-f215c5c44709 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:39.543603 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f4dc9fb2-cfd8-4aed-90c0-f215c5c44709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:39.547592 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd9097-6125-42da-baf1-d2ad009d9682 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:39.554389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e5c01-3903-41cd-a8a5-afcf38c0f873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:39.555392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e5c01-3903-41cd-a8a5-afcf38c0f873 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:39.555574 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e5c01-3903-41cd-a8a5-afcf38c0f873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:39.555933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e5c01-3903-41cd-a8a5-afcf38c0f873 admin admin] PATH_INFO: `/v3/projects/78dc0eb1bf2748dc99ff0bc2a0e9e6be/users/97bdda3b2ec94204a31b30ee535a0f13/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:39.559485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 958/1886] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:39.560400 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc9fb2-cfd8-4aed-90c0-f215c5c44709 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:39.571063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc9fb2-cfd8-4aed-90c0-f215c5c44709 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:39.575797 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bc390-5ab2-4a4d-857e-12d835094c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:39.576460 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bc390-5ab2-4a4d-857e-12d835094c97 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:39.576819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bc390-5ab2-4a4d-857e-12d835094c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:39.577085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e5c01-3903-41cd-a8a5-afcf38c0f873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78dc0eb1bf2748dc99ff0bc2a0e9e6be, user_id=97bdda3b2ec94204a31b30ee535a0f13, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:39.577209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bc390-5ab2-4a4d-857e-12d835094c97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:39.577799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bc390-5ab2-4a4d-857e-12d835094c97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:39.579819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bc390-5ab2-4a4d-857e-12d835094c97 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:39.581082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e5c01-3903-41cd-a8a5-afcf38c0f873 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:39.586034 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:39.608590 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 929/1887] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:39 2022] PUT /identity/v3/projects/78dc0eb1bf2748dc99ff0bc2a0e9e6be/users/97bdda3b2ec94204a31b30ee535a0f13/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:39.610020 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e5c01-3903-41cd-a8a5-afcf38c0f873 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:39.619295 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 959/1888] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:39 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:39.619295 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765bc69-df92-46ae-bc89-c994dff60a8b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:39.619295 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765bc69-df92-46ae-bc89-c994dff60a8b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:39.619295 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765bc69-df92-46ae-bc89-c994dff60a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:39.639492 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:39.650331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0765bc69-df92-46ae-bc89-c994dff60a8b None None] MFA Rules not processed for user `97bdda3b2ec94204a31b30ee535a0f13`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:39.652392 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0765bc69-df92-46ae-bc89-c994dff60a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:39.685557 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e5c01-3903-41cd-a8a5-afcf38c0f873 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:39.734319 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 960/1889] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:39.735474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765bc69-df92-46ae-bc89-c994dff60a8b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:39.750307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a5078-5c9e-4647-9852-d255f0e3afe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:39.751320 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a5078-5c9e-4647-9852-d255f0e3afe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:39.751764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a5078-5c9e-4647-9852-d255f0e3afe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:39.752054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a5078-5c9e-4647-9852-d255f0e3afe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:39.752854 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a5078-5c9e-4647-9852-d255f0e3afe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:39.753343 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a5078-5c9e-4647-9852-d255f0e3afe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:39.754784 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a5078-5c9e-4647-9852-d255f0e3afe8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:39.768056 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 930/1890] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:39.768793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a5078-5c9e-4647-9852-d255f0e3afe8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:39.798972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a5078-5c9e-4647-9852-d255f0e3afe8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:39.806305 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5896e-169d-415d-aab2-6aba4adbca12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:39.807547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5896e-169d-415d-aab2-6aba4adbca12 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:39.807884 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5896e-169d-415d-aab2-6aba4adbca12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:39.808172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a5896e-169d-415d-aab2-6aba4adbca12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:39.809626 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a5896e-169d-415d-aab2-6aba4adbca12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:39.812238 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a5896e-169d-415d-aab2-6aba4adbca12 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:39.818481 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765bc69-df92-46ae-bc89-c994dff60a8b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:39.824294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d975f-0af6-4c19-b25d-dbb811096edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:39.825026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d975f-0af6-4c19-b25d-dbb811096edf admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:39.825317 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d975f-0af6-4c19-b25d-dbb811096edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:39.825596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d975f-0af6-4c19-b25d-dbb811096edf admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:39.826355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d975f-0af6-4c19-b25d-dbb811096edf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:39.828538 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d975f-0af6-4c19-b25d-dbb811096edf admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:39.846959 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 931/1891] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:39 2022] POST /identity/v3/projects => generated 387 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:39.848816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5896e-169d-415d-aab2-6aba4adbca12 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:39.860581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 961/1892] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:39 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:39.861832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d975f-0af6-4c19-b25d-dbb811096edf admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:39.875959 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d975f-0af6-4c19-b25d-dbb811096edf admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:39.920875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a5896e-169d-415d-aab2-6aba4adbca12 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:39.925854 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fd728-d4b5-4699-bf77-1f2a442c0079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:39.926522 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fd728-d4b5-4699-bf77-1f2a442c0079 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:39.926805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fd728-d4b5-4699-bf77-1f2a442c0079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:39.927082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fd728-d4b5-4699-bf77-1f2a442c0079 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:39.928114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88fd728-d4b5-4699-bf77-1f2a442c0079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:39.929751 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88fd728-d4b5-4699-bf77-1f2a442c0079 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:39.948909 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79980c1e-2a25-4803-8896-e066ff2bb738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:39.949169 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79980c1e-2a25-4803-8896-e066ff2bb738 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:39.949588 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79980c1e-2a25-4803-8896-e066ff2bb738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:39.949829 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79980c1e-2a25-4803-8896-e066ff2bb738 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:39.950622 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79980c1e-2a25-4803-8896-e066ff2bb738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:39.952797 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79980c1e-2a25-4803-8896-e066ff2bb738 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:39.956246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:39.964747 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 932/1893] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:39 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:39.966564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fd728-d4b5-4699-bf77-1f2a442c0079 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:39.979725 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 962/1894] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:39 2022] POST /identity/v3/users => generated 300 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:39.981558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79980c1e-2a25-4803-8896-e066ff2bb738 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:39.991764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79980c1e-2a25-4803-8896-e066ff2bb738 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:40.010166 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:40.037841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fd728-d4b5-4699-bf77-1f2a442c0079 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:40.042555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48f13a-8b59-406f-9037-7a54453bf98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:40.043218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48f13a-8b59-406f-9037-7a54453bf98b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:40.043506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48f13a-8b59-406f-9037-7a54453bf98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:40.043781 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48f13a-8b59-406f-9037-7a54453bf98b admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:40.044550 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48f13a-8b59-406f-9037-7a54453bf98b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:40.046146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48f13a-8b59-406f-9037-7a54453bf98b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:40.051717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1675f6-37f5-422c-b454-364b7f181f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:40.052408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1675f6-37f5-422c-b454-364b7f181f2c admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:40.052727 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1675f6-37f5-422c-b454-364b7f181f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:40.053009 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1675f6-37f5-422c-b454-364b7f181f2c admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:40.053708 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1675f6-37f5-422c-b454-364b7f181f2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:40.055639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1675f6-37f5-422c-b454-364b7f181f2c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:40.058410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 933/1895] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:40.059175 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48f13a-8b59-406f-9037-7a54453bf98b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:40.071060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48f13a-8b59-406f-9037-7a54453bf98b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:40.076567 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08836d-8c4e-4a71-a808-9a2f49df6db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:40.077243 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08836d-8c4e-4a71-a808-9a2f49df6db1 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:40.077534 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08836d-8c4e-4a71-a808-9a2f49df6db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:40.077920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08836d-8c4e-4a71-a808-9a2f49df6db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:40.078658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08836d-8c4e-4a71-a808-9a2f49df6db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:40.080303 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08836d-8c4e-4a71-a808-9a2f49df6db1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:40.083366 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 963/1896] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:39 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:40.084934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1675f6-37f5-422c-b454-364b7f181f2c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:40.094485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 934/1897] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:40.095331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08836d-8c4e-4a71-a808-9a2f49df6db1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:40.096876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1675f6-37f5-422c-b454-364b7f181f2c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:40.101906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25916039-1044-4514-b00d-bf802bc4acc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:40.102698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25916039-1044-4514-b00d-bf802bc4acc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:40.103102 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25916039-1044-4514-b00d-bf802bc4acc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:40.103559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25916039-1044-4514-b00d-bf802bc4acc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:40.104596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25916039-1044-4514-b00d-bf802bc4acc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:40.105879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08836d-8c4e-4a71-a808-9a2f49df6db1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:40.106210 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25916039-1044-4514-b00d-bf802bc4acc0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:40.110739 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19122b39-1fa6-4984-a83c-d688b099c406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:40.111465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19122b39-1fa6-4984-a83c-d688b099c406 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:40.111749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19122b39-1fa6-4984-a83c-d688b099c406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:40.112029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19122b39-1fa6-4984-a83c-d688b099c406 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:40.112823 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19122b39-1fa6-4984-a83c-d688b099c406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:40.114411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19122b39-1fa6-4984-a83c-d688b099c406 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:40.117389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 964/1898] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:40.118285 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25916039-1044-4514-b00d-bf802bc4acc0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:40.126115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 935/1899] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:40.126905 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19122b39-1fa6-4984-a83c-d688b099c406 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:40.132106 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25916039-1044-4514-b00d-bf802bc4acc0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:40.136788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6027f0-f151-4736-b0c3-4e9e7da5dfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:40.137012 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19122b39-1fa6-4984-a83c-d688b099c406 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:40.137672 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6027f0-f151-4736-b0c3-4e9e7da5dfcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:40.137951 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6027f0-f151-4736-b0c3-4e9e7da5dfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:40.138337 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6027f0-f151-4736-b0c3-4e9e7da5dfcc admin admin] PATH_INFO: `/v3/projects/84d0ff2656794b17a70cbb92ac152297/users/a1fe7fe70d8b4c6c930a6c6c0410debf/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:40.142069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517a10b-90a0-4e0b-be6f-a4566c11008b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:40.142925 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517a10b-90a0-4e0b-be6f-a4566c11008b admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:40.143325 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517a10b-90a0-4e0b-be6f-a4566c11008b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:40.143605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517a10b-90a0-4e0b-be6f-a4566c11008b admin admin] PATH_INFO: `/v3/projects/e2b9d43ca5a4485cbe4c85ab26e439f4/users/3a7fa4b4092d4238a70068156d4a2f94/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:40.153899 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6027f0-f151-4736-b0c3-4e9e7da5dfcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=84d0ff2656794b17a70cbb92ac152297, user_id=a1fe7fe70d8b4c6c930a6c6c0410debf, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:40.156551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6027f0-f151-4736-b0c3-4e9e7da5dfcc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:40.158927 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517a10b-90a0-4e0b-be6f-a4566c11008b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b9d43ca5a4485cbe4c85ab26e439f4, user_id=3a7fa4b4092d4238a70068156d4a2f94, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:40.161710 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517a10b-90a0-4e0b-be6f-a4566c11008b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:40.170135 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 965/1900] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:40 2022] PUT /identity/v3/projects/84d0ff2656794b17a70cbb92ac152297/users/a1fe7fe70d8b4c6c930a6c6c0410debf/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:40.170704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6027f0-f151-4736-b0c3-4e9e7da5dfcc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:40.177742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 936/1901] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:40 2022] PUT /identity/v3/projects/e2b9d43ca5a4485cbe4c85ab26e439f4/users/3a7fa4b4092d4238a70068156d4a2f94/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:40.179358 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517a10b-90a0-4e0b-be6f-a4566c11008b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:40.238656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6027f0-f151-4736-b0c3-4e9e7da5dfcc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:40.243390 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e6a4e-d15d-4a10-a820-a7aa3eb9ff41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:40.244058 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e6a4e-d15d-4a10-a820-a7aa3eb9ff41 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:40.244344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e6a4e-d15d-4a10-a820-a7aa3eb9ff41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:40.244657 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e6a4e-d15d-4a10-a820-a7aa3eb9ff41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:40.245185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517a10b-90a0-4e0b-be6f-a4566c11008b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:40.245386 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e6a4e-d15d-4a10-a820-a7aa3eb9ff41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:40.247000 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e6a4e-d15d-4a10-a820-a7aa3eb9ff41 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:40.249820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c336946-2a21-4d44-a630-4397f7f02159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:40.250487 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c336946-2a21-4d44-a630-4397f7f02159 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:40.250772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c336946-2a21-4d44-a630-4397f7f02159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:40.251052 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c336946-2a21-4d44-a630-4397f7f02159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:40.251775 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c336946-2a21-4d44-a630-4397f7f02159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:40.253397 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c336946-2a21-4d44-a630-4397f7f02159 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:40.267432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 937/1902] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:40.267432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209149d-c626-4feb-a542-e692845ae7dc None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:40.267432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209149d-c626-4feb-a542-e692845ae7dc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:40.267432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209149d-c626-4feb-a542-e692845ae7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:40.269007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 966/1903] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:40.272665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e6a4e-d15d-4a10-a820-a7aa3eb9ff41 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:40.284461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e6a4e-d15d-4a10-a820-a7aa3eb9ff41 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:40.289490 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67281593-ef70-45ac-8740-deef2514c8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:40.290859 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67281593-ef70-45ac-8740-deef2514c8d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:40.290859 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67281593-ef70-45ac-8740-deef2514c8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:40.291117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67281593-ef70-45ac-8740-deef2514c8d8 admin admin] PATH_INFO: `/v3/projects/84d0ff2656794b17a70cbb92ac152297/users/a1fe7fe70d8b4c6c930a6c6c0410debf/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:40.293190 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67281593-ef70-45ac-8740-deef2514c8d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84d0ff2656794b17a70cbb92ac152297, user_id=a1fe7fe70d8b4c6c930a6c6c0410debf, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:40.295942 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67281593-ef70-45ac-8740-deef2514c8d8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:40.307497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 967/1904] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:40 2022] PUT /identity/v3/projects/84d0ff2656794b17a70cbb92ac152297/users/a1fe7fe70d8b4c6c930a6c6c0410debf/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:40.308537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67281593-ef70-45ac-8740-deef2514c8d8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:40.378167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f209149d-c626-4feb-a542-e692845ae7dc None None] MFA Rules not processed for user `3a7fa4b4092d4238a70068156d4a2f94`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:40.381994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f209149d-c626-4feb-a542-e692845ae7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:40.452235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:40.456346 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67281593-ef70-45ac-8740-deef2514c8d8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:40.462513 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a1d80-3b82-4974-8523-ad8ecf67aa56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:40.463526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a1d80-3b82-4974-8523-ad8ecf67aa56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:40.463818 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a1d80-3b82-4974-8523-ad8ecf67aa56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:40.464052 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:40.464434 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a1d80-3b82-4974-8523-ad8ecf67aa56 admin admin] PATH_INFO: `/v3/projects/9b76defec9564410aacfede3c9420966/users/06fdc2590ce3471ab58d902957305606/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:40.483692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a1d80-3b82-4974-8523-ad8ecf67aa56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b76defec9564410aacfede3c9420966, user_id=06fdc2590ce3471ab58d902957305606, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:40.486451 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a1d80-3b82-4974-8523-ad8ecf67aa56 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:40.508977 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 968/1905] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:40 2022] PUT /identity/v3/projects/9b76defec9564410aacfede3c9420966/users/06fdc2590ce3471ab58d902957305606/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:40.520611 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe664b-65f6-428d-9101-c4fb152d0204 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:40.520963 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe664b-65f6-428d-9101-c4fb152d0204 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:40.521256 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe664b-65f6-428d-9101-c4fb152d0204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:40.525692 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 938/1906] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:40.527587 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf0a94-cbfb-4726-accd-b8dccd0f8f87 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:40.528104 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf0a94-cbfb-4726-accd-b8dccd0f8f87 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:40.528854 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf0a94-cbfb-4726-accd-b8dccd0f8f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:40.556376 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3efe664b-65f6-428d-9101-c4fb152d0204 None None] MFA Rules not processed for user `a1fe7fe70d8b4c6c930a6c6c0410debf`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:40.557285 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3efe664b-65f6-428d-9101-c4fb152d0204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:40.567951 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4daf0a94-cbfb-4726-accd-b8dccd0f8f87 None None] MFA Rules not processed for user `06fdc2590ce3471ab58d902957305606`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:40.568872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4daf0a94-cbfb-4726-accd-b8dccd0f8f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:40.685172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 939/1907] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:40.686984 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf0a94-cbfb-4726-accd-b8dccd0f8f87 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:40.692089 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 969/1908] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:40.693294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe664b-65f6-428d-9101-c4fb152d0204 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:40.769089 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf0a94-cbfb-4726-accd-b8dccd0f8f87 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:40.779099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d1565-0d43-4a01-bb2f-9056b866f906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:40.779979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d1565-0d43-4a01-bb2f-9056b866f906 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:40.780412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d1565-0d43-4a01-bb2f-9056b866f906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:40.780727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d1565-0d43-4a01-bb2f-9056b866f906 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:40.781548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481d1565-0d43-4a01-bb2f-9056b866f906 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:40.783644 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe664b-65f6-428d-9101-c4fb152d0204 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:40.784748 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481d1565-0d43-4a01-bb2f-9056b866f906 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:40.789948 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e769b-1f22-4ebb-bc22-03e437ab4834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:40.790897 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e769b-1f22-4ebb-bc22-03e437ab4834 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:40.791378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e769b-1f22-4ebb-bc22-03e437ab4834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:40.791851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e769b-1f22-4ebb-bc22-03e437ab4834 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:40.792795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e769b-1f22-4ebb-bc22-03e437ab4834 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:40.795796 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e769b-1f22-4ebb-bc22-03e437ab4834 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:40.824479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 940/1909] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:40 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:40.825456 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d1565-0d43-4a01-bb2f-9056b866f906 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:40.828652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-481d1565-0d43-4a01-bb2f-9056b866f906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:40.844070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 970/1910] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:40 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:40.845129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e769b-1f22-4ebb-bc22-03e437ab4834 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:40.931723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d1565-0d43-4a01-bb2f-9056b866f906 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:40.940111 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:40.962711 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:40.969286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e769b-1f22-4ebb-bc22-03e437ab4834 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:40.982602 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfe94c-28ba-401b-a8cb-ee3eb887a2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:40.984935 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfe94c-28ba-401b-a8cb-ee3eb887a2e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:40.987594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfe94c-28ba-401b-a8cb-ee3eb887a2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:40.988225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfe94c-28ba-401b-a8cb-ee3eb887a2e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:40.990041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfe94c-28ba-401b-a8cb-ee3eb887a2e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:40.992695 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfe94c-28ba-401b-a8cb-ee3eb887a2e8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:41.035662 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 971/1911] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:40 2022] POST /identity/v3/users => generated 296 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:41.036652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfe94c-28ba-401b-a8cb-ee3eb887a2e8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:41.064086 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfe94c-28ba-401b-a8cb-ee3eb887a2e8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:41.076569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff0e31-98dd-478a-8c9f-0b21e9c4072c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:41.079023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff0e31-98dd-478a-8c9f-0b21e9c4072c admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:41.079441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff0e31-98dd-478a-8c9f-0b21e9c4072c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:41.080048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff0e31-98dd-478a-8c9f-0b21e9c4072c admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:41.081485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff0e31-98dd-478a-8c9f-0b21e9c4072c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:41.087666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff0e31-98dd-478a-8c9f-0b21e9c4072c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:41.091611 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d9522-a574-4397-8087-b1f92e4e13a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:41.094515 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d9522-a574-4397-8087-b1f92e4e13a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:41.094877 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d9522-a574-4397-8087-b1f92e4e13a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:41.095228 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d9522-a574-4397-8087-b1f92e4e13a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:41.137305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 972/1912] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:41 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:41.138914 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff0e31-98dd-478a-8c9f-0b21e9c4072c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:41.163525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff0e31-98dd-478a-8c9f-0b21e9c4072c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:41.174272 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24573d0c-14d9-4c78-b7b5-c9b7ddf059cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:41.182405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24573d0c-14d9-4c78-b7b5-c9b7ddf059cf admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:41.182924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24573d0c-14d9-4c78-b7b5-c9b7ddf059cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:41.183354 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24573d0c-14d9-4c78-b7b5-c9b7ddf059cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:41.185506 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24573d0c-14d9-4c78-b7b5-c9b7ddf059cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:41.187369 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24573d0c-14d9-4c78-b7b5-c9b7ddf059cf admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:41.201479 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 973/1913] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:41.202612 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24573d0c-14d9-4c78-b7b5-c9b7ddf059cf admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:41.218114 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24573d0c-14d9-4c78-b7b5-c9b7ddf059cf admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:41.224054 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f5583-0581-43ee-bf30-8fc5533a4227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:41.225141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f5583-0581-43ee-bf30-8fc5533a4227 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:41.225758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f5583-0581-43ee-bf30-8fc5533a4227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:41.227090 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f5583-0581-43ee-bf30-8fc5533a4227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:41.227986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f5583-0581-43ee-bf30-8fc5533a4227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:41.230807 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f5583-0581-43ee-bf30-8fc5533a4227 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:41.238545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d9522-a574-4397-8087-b1f92e4e13a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:41.241180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d9522-a574-4397-8087-b1f92e4e13a8 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:41.253354 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 974/1914] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:41.256419 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f5583-0581-43ee-bf30-8fc5533a4227 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:41.262597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 941/1915] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:41.263412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d9522-a574-4397-8087-b1f92e4e13a8 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:41.274406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d9522-a574-4397-8087-b1f92e4e13a8 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:41.280676 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4b5e9-0476-42cb-bad3-379fa8afada6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:41.281638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4b5e9-0476-42cb-bad3-379fa8afada6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:41.281991 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4b5e9-0476-42cb-bad3-379fa8afada6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:41.282337 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4b5e9-0476-42cb-bad3-379fa8afada6 admin admin] PATH_INFO: `/v3/projects/f3c4986a06a74eb2b0dbdde79a35c482/users/f612d09ec9c64f02aaacd514696642cd/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:41.294501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f5583-0581-43ee-bf30-8fc5533a4227 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:41.306829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4b5e9-0476-42cb-bad3-379fa8afada6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3c4986a06a74eb2b0dbdde79a35c482, user_id=f612d09ec9c64f02aaacd514696642cd, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:41.309944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4b5e9-0476-42cb-bad3-379fa8afada6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:41.314783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026888f-d4ca-491a-9dfd-3324344a6de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:41.315819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026888f-d4ca-491a-9dfd-3324344a6de7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:41.316815 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026888f-d4ca-491a-9dfd-3324344a6de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:41.316999 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026888f-d4ca-491a-9dfd-3324344a6de7 admin admin] PATH_INFO: `/v3/projects/4dc2b1805aa740ef87e8eaaac3815228/users/3c7c65c395e54952b332f390c6285a5f/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:41.339598 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e026888f-d4ca-491a-9dfd-3324344a6de7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dc2b1805aa740ef87e8eaaac3815228, user_id=3c7c65c395e54952b332f390c6285a5f, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:41.342160 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 942/1916] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:41 2022] PUT /identity/v3/projects/f3c4986a06a74eb2b0dbdde79a35c482/users/f612d09ec9c64f02aaacd514696642cd/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:41.343133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4b5e9-0476-42cb-bad3-379fa8afada6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:41.343272 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e026888f-d4ca-491a-9dfd-3324344a6de7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:41.373101 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 975/1917] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:41 2022] PUT /identity/v3/projects/4dc2b1805aa740ef87e8eaaac3815228/users/3c7c65c395e54952b332f390c6285a5f/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:41.373964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026888f-d4ca-491a-9dfd-3324344a6de7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:41.482718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4b5e9-0476-42cb-bad3-379fa8afada6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:41.483250 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026888f-d4ca-491a-9dfd-3324344a6de7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:41.487567 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d9439-ca9e-4f95-80cf-4092da9ef87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:41.488643 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a6e5f-1fe7-4f59-936e-4d46c78275f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:41.489025 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d9439-ca9e-4f95-80cf-4092da9ef87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:41.489400 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d9439-ca9e-4f95-80cf-4092da9ef87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:41.489510 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a6e5f-1fe7-4f59-936e-4d46c78275f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:41.489772 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a6e5f-1fe7-4f59-936e-4d46c78275f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:41.489875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d9439-ca9e-4f95-80cf-4092da9ef87e admin admin] PATH_INFO: `/v3/users/97bdda3b2ec94204a31b30ee535a0f13` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:41.490051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a6e5f-1fe7-4f59-936e-4d46c78275f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:41.490815 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a6e5f-1fe7-4f59-936e-4d46c78275f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:41.490971 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d9439-ca9e-4f95-80cf-4092da9ef87e admin admin] RBAC: Authorizing `identity:delete_user(user_id=97bdda3b2ec94204a31b30ee535a0f13)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:41.492423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a6e5f-1fe7-4f59-936e-4d46c78275f7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:41.492941 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d9439-ca9e-4f95-80cf-4092da9ef87e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:41.505763 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 976/1918] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:41.506639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a6e5f-1fe7-4f59-936e-4d46c78275f7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:41.519810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a6e5f-1fe7-4f59-936e-4d46c78275f7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:41.525523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313705d-f1dc-41e9-a344-8d013da01e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:41.526197 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313705d-f1dc-41e9-a344-8d013da01e05 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:41.526483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313705d-f1dc-41e9-a344-8d013da01e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:41.526759 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313705d-f1dc-41e9-a344-8d013da01e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:41.527494 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313705d-f1dc-41e9-a344-8d013da01e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:41.529138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313705d-f1dc-41e9-a344-8d013da01e05 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:41.538970 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:41.545257 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 977/1919] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:41.546513 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313705d-f1dc-41e9-a344-8d013da01e05 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:41.553515 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8a9d9439-ca9e-4f95-80cf-4092da9ef87e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97bdda3b2ec94204a31b30ee535a0f13'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:41.554540 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8a9d9439-ca9e-4f95-80cf-4092da9ef87e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97bdda3b2ec94204a31b30ee535a0f13'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:41.555342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8a9d9439-ca9e-4f95-80cf-4092da9ef87e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97bdda3b2ec94204a31b30ee535a0f13'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:41.558423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313705d-f1dc-41e9-a344-8d013da01e05 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:41.564051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea889f-f423-4f22-8136-01525355d9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:41.564532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8a9d9439-ca9e-4f95-80cf-4092da9ef87e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97bdda3b2ec94204a31b30ee535a0f13'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:41.565319 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea889f-f423-4f22-8136-01525355d9b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:41.565433 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea889f-f423-4f22-8136-01525355d9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:41.565747 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea889f-f423-4f22-8136-01525355d9b6 admin admin] PATH_INFO: `/v3/projects/f3c4986a06a74eb2b0dbdde79a35c482/users/f612d09ec9c64f02aaacd514696642cd/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:41.567912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea889f-f423-4f22-8136-01525355d9b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3c4986a06a74eb2b0dbdde79a35c482, user_id=f612d09ec9c64f02aaacd514696642cd, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:41.570723 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea889f-f423-4f22-8136-01525355d9b6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:41.579843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8a9d9439-ca9e-4f95-80cf-4092da9ef87e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97bdda3b2ec94204a31b30ee535a0f13'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:41.583787 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 978/1920] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:41 2022] PUT /identity/v3/projects/f3c4986a06a74eb2b0dbdde79a35c482/users/f612d09ec9c64f02aaacd514696642cd/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:41.584617 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea889f-f423-4f22-8136-01525355d9b6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:41.586269 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1eea889f-f423-4f22-8136-01525355d9b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:41.606419 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 943/1921] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:41 2022] DELETE /identity/v3/users/97bdda3b2ec94204a31b30ee535a0f13 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:41.610613 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b96dc-dc04-4fb7-b70a-e521d1abc466 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:41.611017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b96dc-dc04-4fb7-b70a-e521d1abc466 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:41.611713 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b96dc-dc04-4fb7-b70a-e521d1abc466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:41.645980 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:41.651333 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-136b96dc-dc04-4fb7-b70a-e521d1abc466 None None] MFA Rules not processed for user `f612d09ec9c64f02aaacd514696642cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:41.655015 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-136b96dc-dc04-4fb7-b70a-e521d1abc466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:41.670672 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea889f-f423-4f22-8136-01525355d9b6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:41.758896 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 944/1922] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:41.760332 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f891a-3ebd-4400-abf4-008b947e2e4e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:41.760657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f891a-3ebd-4400-abf4-008b947e2e4e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:41.760938 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f891a-3ebd-4400-abf4-008b947e2e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:41.778314 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9de38-4469-4e85-a068-e90285f4f1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:41.780131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9de38-4469-4e85-a068-e90285f4f1e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:41.780449 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9de38-4469-4e85-a068-e90285f4f1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:41.780761 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9de38-4469-4e85-a068-e90285f4f1e2 admin admin] PATH_INFO: `/v3/projects/4dc2b1805aa740ef87e8eaaac3815228/users/3c7c65c395e54952b332f390c6285a5f/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:41.782815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-029f891a-3ebd-4400-abf4-008b947e2e4e None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:41.782994 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9de38-4469-4e85-a068-e90285f4f1e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dc2b1805aa740ef87e8eaaac3815228, user_id=3c7c65c395e54952b332f390c6285a5f, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:41.784217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-029f891a-3ebd-4400-abf4-008b947e2e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:41.787353 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9de38-4469-4e85-a068-e90285f4f1e2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:41.799324 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 945/1923] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:41.800636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f891a-3ebd-4400-abf4-008b947e2e4e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:41.802054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-029f891a-3ebd-4400-abf4-008b947e2e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:41.812740 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 979/1924] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:41 2022] PUT /identity/v3/projects/4dc2b1805aa740ef87e8eaaac3815228/users/3c7c65c395e54952b332f390c6285a5f/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:41.814382 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9de38-4469-4e85-a068-e90285f4f1e2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:41.816984 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a0d9de38-4469-4e85-a068-e90285f4f1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:41.879474 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f891a-3ebd-4400-abf4-008b947e2e4e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:41.904400 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9de38-4469-4e85-a068-e90285f4f1e2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:41.910854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd908af-8f86-434f-9f69-c1e9869707e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:41.911248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd908af-8f86-434f-9f69-c1e9869707e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:41.911660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd908af-8f86-434f-9f69-c1e9869707e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:41.911940 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd908af-8f86-434f-9f69-c1e9869707e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:41.933260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:41.943757 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf52ec-932a-4e2a-ad22-fd7a57f5fa40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:41.944831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf52ec-932a-4e2a-ad22-fd7a57f5fa40 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:41.945877 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf52ec-932a-4e2a-ad22-fd7a57f5fa40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:41.946510 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf52ec-932a-4e2a-ad22-fd7a57f5fa40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:42.015047 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd908af-8f86-434f-9f69-c1e9869707e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:42.016856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd908af-8f86-434f-9f69-c1e9869707e0 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:42.029961 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 980/1925] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:42.031221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67f798-e541-4151-8edc-94733692a91d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:42.031679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67f798-e541-4151-8edc-94733692a91d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:42.032026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67f798-e541-4151-8edc-94733692a91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:42.038481 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf52ec-932a-4e2a-ad22-fd7a57f5fa40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:42.040975 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf52ec-932a-4e2a-ad22-fd7a57f5fa40 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:42.054283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 946/1926] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:42.055148 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf52ec-932a-4e2a-ad22-fd7a57f5fa40 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:42.056456 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4fbf52ec-932a-4e2a-ad22-fd7a57f5fa40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:42.063894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ad67f798-e541-4151-8edc-94733692a91d None None] MFA Rules not processed for user `3c7c65c395e54952b332f390c6285a5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:42.065214 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ad67f798-e541-4151-8edc-94733692a91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:42.072834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf52ec-932a-4e2a-ad22-fd7a57f5fa40 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:42.079882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1acd39-4c0b-4005-8cc0-20dbd2c96746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:42.081103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1acd39-4c0b-4005-8cc0-20dbd2c96746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:42.082050 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1acd39-4c0b-4005-8cc0-20dbd2c96746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:42.082410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1acd39-4c0b-4005-8cc0-20dbd2c96746 admin admin] PATH_INFO: `/v3/users/9a5173d73c9a475c998a586eca7bb913` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:42.083877 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1acd39-4c0b-4005-8cc0-20dbd2c96746 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a5173d73c9a475c998a586eca7bb913)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:42.085969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1acd39-4c0b-4005-8cc0-20dbd2c96746 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:42.103501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:42.167587 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 981/1927] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:42 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:42.170997 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cf1acd39-4c0b-4005-8cc0-20dbd2c96746 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a5173d73c9a475c998a586eca7bb913'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.171963 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cf1acd39-4c0b-4005-8cc0-20dbd2c96746 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a5173d73c9a475c998a586eca7bb913'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.172745 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cf1acd39-4c0b-4005-8cc0-20dbd2c96746 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a5173d73c9a475c998a586eca7bb913'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.180399 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67f798-e541-4151-8edc-94733692a91d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:42.192110 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67f798-e541-4151-8edc-94733692a91d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:42.197297 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cf1acd39-4c0b-4005-8cc0-20dbd2c96746 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a5173d73c9a475c998a586eca7bb913'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.202052 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11048809-f546-44e3-a847-7405ad48ec34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:42.202755 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11048809-f546-44e3-a847-7405ad48ec34 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:42.203514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11048809-f546-44e3-a847-7405ad48ec34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:42.203892 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11048809-f546-44e3-a847-7405ad48ec34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:42.205924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-11048809-f546-44e3-a847-7405ad48ec34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:42.213090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cf1acd39-4c0b-4005-8cc0-20dbd2c96746 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a5173d73c9a475c998a586eca7bb913'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.223977 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11048809-f546-44e3-a847-7405ad48ec34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:42.232810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11048809-f546-44e3-a847-7405ad48ec34 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:42.234649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 947/1928] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:42 2022] DELETE /identity/v3/users/9a5173d73c9a475c998a586eca7bb913 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:42.235807 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1acd39-4c0b-4005-8cc0-20dbd2c96746 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:42.237315 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cf1acd39-4c0b-4005-8cc0-20dbd2c96746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:42.239999 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 982/1929] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:42.242620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0115290-d29d-4a89-9187-51458db164c4 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:42.242923 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0115290-d29d-4a89-9187-51458db164c4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:42.243205 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0115290-d29d-4a89-9187-51458db164c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:42.266344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c0115290-d29d-4a89-9187-51458db164c4 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:42.267474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c0115290-d29d-4a89-9187-51458db164c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:42.345098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 983/1930] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:42.357104 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1acd39-4c0b-4005-8cc0-20dbd2c96746 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:42.362164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36684178-5c4e-4c58-9024-59659bc239a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:42.362500 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36684178-5c4e-4c58-9024-59659bc239a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:42.364398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36684178-5c4e-4c58-9024-59659bc239a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:42.364398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36684178-5c4e-4c58-9024-59659bc239a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:42.365864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-36684178-5c4e-4c58-9024-59659bc239a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:42.370630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0115290-d29d-4a89-9187-51458db164c4 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:42.372277 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c0115290-d29d-4a89-9187-51458db164c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:42.400946 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:42.491525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0115290-d29d-4a89-9187-51458db164c4 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:42.499768 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a2185-e4d0-42d8-86f6-4ecbd0cb6bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:42.500458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a2185-e4d0-42d8-86f6-4ecbd0cb6bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:42.500925 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a2185-e4d0-42d8-86f6-4ecbd0cb6bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:42.501098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a2185-e4d0-42d8-86f6-4ecbd0cb6bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:42.512906 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36684178-5c4e-4c58-9024-59659bc239a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:42.518965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a2185-e4d0-42d8-86f6-4ecbd0cb6bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:42.520807 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a2185-e4d0-42d8-86f6-4ecbd0cb6bc6 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:42.522798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36684178-5c4e-4c58-9024-59659bc239a0 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:42.536250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 948/1931] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:42.537510 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36684178-5c4e-4c58-9024-59659bc239a0 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:42.538857 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-36684178-5c4e-4c58-9024-59659bc239a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:42.543383 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 984/1932] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:42.555209 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36684178-5c4e-4c58-9024-59659bc239a0 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:42.560862 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d330d-fb26-466d-93ee-a551bd8f612e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:42.562729 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d330d-fb26-466d-93ee-a551bd8f612e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:42.563020 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d330d-fb26-466d-93ee-a551bd8f612e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:42.563294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d330d-fb26-466d-93ee-a551bd8f612e admin admin] PATH_INFO: `/v3/projects/78dc0eb1bf2748dc99ff0bc2a0e9e6be` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:42.565402 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d330d-fb26-466d-93ee-a551bd8f612e admin admin] RBAC: Authorizing `identity:delete_project(project_id=78dc0eb1bf2748dc99ff0bc2a0e9e6be)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:42.568247 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d330d-fb26-466d-93ee-a551bd8f612e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:42.650289 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-602d330d-fb26-466d-93ee-a551bd8f612e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78dc0eb1bf2748dc99ff0bc2a0e9e6be', 'request_id': 'req-602d330d-fb26-466d-93ee-a551bd8f612e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.665854 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-602d330d-fb26-466d-93ee-a551bd8f612e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78dc0eb1bf2748dc99ff0bc2a0e9e6be', 'request_id': 'req-602d330d-fb26-466d-93ee-a551bd8f612e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.681220 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-602d330d-fb26-466d-93ee-a551bd8f612e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78dc0eb1bf2748dc99ff0bc2a0e9e6be', 'request_id': 'req-602d330d-fb26-466d-93ee-a551bd8f612e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.708593 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-602d330d-fb26-466d-93ee-a551bd8f612e admin admin] The token cache is being invalidate because project 78dc0eb1bf2748dc99ff0bc2a0e9e6be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:42.708894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-602d330d-fb26-466d-93ee-a551bd8f612e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.709608 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-602d330d-fb26-466d-93ee-a551bd8f612e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.711545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 949/1933] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:42 2022] DELETE /identity/v3/projects/78dc0eb1bf2748dc99ff0bc2a0e9e6be => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:42.715271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a2185-e4d0-42d8-86f6-4ecbd0cb6bc6 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:42.727271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a2185-e4d0-42d8-86f6-4ecbd0cb6bc6 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:42.796325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:42.812201 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11674f64-1bba-44d1-acb2-99bb0ff9b4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:42.812408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11674f64-1bba-44d1-acb2-99bb0ff9b4bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:42.812616 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11674f64-1bba-44d1-acb2-99bb0ff9b4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:42.812989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d330d-fb26-466d-93ee-a551bd8f612e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:42.813112 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11674f64-1bba-44d1-acb2-99bb0ff9b4bb admin admin] PATH_INFO: `/v3/users/a1fe7fe70d8b4c6c930a6c6c0410debf` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:42.815332 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-602d330d-fb26-466d-93ee-a551bd8f612e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:42.821393 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11674f64-1bba-44d1-acb2-99bb0ff9b4bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1fe7fe70d8b4c6c930a6c6c0410debf)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:42.825657 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11674f64-1bba-44d1-acb2-99bb0ff9b4bb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:42.835063 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d330d-fb26-466d-93ee-a551bd8f612e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:42.840356 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c38e9-c354-4018-a727-c97765300415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:42.841394 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c38e9-c354-4018-a727-c97765300415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:42.841789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c38e9-c354-4018-a727-c97765300415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:42.842400 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c38e9-c354-4018-a727-c97765300415 admin admin] PATH_INFO: `/v3/users/3a7fa4b4092d4238a70068156d4a2f94` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:42.843703 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c38e9-c354-4018-a727-c97765300415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a7fa4b4092d4238a70068156d4a2f94)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:42.846132 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c38e9-c354-4018-a727-c97765300415 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:42.902038 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-11674f64-1bba-44d1-acb2-99bb0ff9b4bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1fe7fe70d8b4c6c930a6c6c0410debf'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.913261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ad9c38e9-c354-4018-a727-c97765300415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a7fa4b4092d4238a70068156d4a2f94'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.914204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ad9c38e9-c354-4018-a727-c97765300415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a7fa4b4092d4238a70068156d4a2f94'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.914827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ad9c38e9-c354-4018-a727-c97765300415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a7fa4b4092d4238a70068156d4a2f94'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.925990 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ad9c38e9-c354-4018-a727-c97765300415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a7fa4b4092d4238a70068156d4a2f94'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.926669 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-11674f64-1bba-44d1-acb2-99bb0ff9b4bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1fe7fe70d8b4c6c930a6c6c0410debf'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.927642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-11674f64-1bba-44d1-acb2-99bb0ff9b4bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1fe7fe70d8b4c6c930a6c6c0410debf'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.928333 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-11674f64-1bba-44d1-acb2-99bb0ff9b4bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1fe7fe70d8b4c6c930a6c6c0410debf'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.936560 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-11674f64-1bba-44d1-acb2-99bb0ff9b4bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1fe7fe70d8b4c6c930a6c6c0410debf'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.938915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ad9c38e9-c354-4018-a727-c97765300415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a7fa4b4092d4238a70068156d4a2f94'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:42.953523 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 985/1934] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:42 2022] DELETE /identity/v3/users/a1fe7fe70d8b4c6c930a6c6c0410debf => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:29:42.955350 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11674f64-1bba-44d1-acb2-99bb0ff9b4bb admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:42.957954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-11674f64-1bba-44d1-acb2-99bb0ff9b4bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:42.981784 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 950/1935] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:42 2022] DELETE /identity/v3/users/3a7fa4b4092d4238a70068156d4a2f94 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:42.983485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bef651-bf99-4e2c-af6c-3decbeb22fab None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:42.983778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bef651-bf99-4e2c-af6c-3decbeb22fab None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:42.984055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bef651-bf99-4e2c-af6c-3decbeb22fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:43.015014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-81bef651-bf99-4e2c-af6c-3decbeb22fab None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:43.016340 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-81bef651-bf99-4e2c-af6c-3decbeb22fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:43.037947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:43.134261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 951/1936] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:43.135265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bef651-bf99-4e2c-af6c-3decbeb22fab None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:43.136714 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-81bef651-bf99-4e2c-af6c-3decbeb22fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:43.147150 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11674f64-1bba-44d1-acb2-99bb0ff9b4bb admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:43.155445 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bef651-bf99-4e2c-af6c-3decbeb22fab None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:43.159465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c605552-525e-41c7-b5f9-95e4128e4d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:43.161594 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c605552-525e-41c7-b5f9-95e4128e4d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:43.161897 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c605552-525e-41c7-b5f9-95e4128e4d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:43.162184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c605552-525e-41c7-b5f9-95e4128e4d6c admin admin] PATH_INFO: `/v3/projects/310f1a2f1f84472c9b611bbcf002da26` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:43.165700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c605552-525e-41c7-b5f9-95e4128e4d6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=310f1a2f1f84472c9b611bbcf002da26)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:43.166131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2443e1-8c18-460f-a8fa-e4c59f42d55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:43.167558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2443e1-8c18-460f-a8fa-e4c59f42d55a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:43.167926 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2443e1-8c18-460f-a8fa-e4c59f42d55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:43.168186 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c605552-525e-41c7-b5f9-95e4128e4d6c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:43.168291 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2443e1-8c18-460f-a8fa-e4c59f42d55a admin admin] PATH_INFO: `/v3/users/06fdc2590ce3471ab58d902957305606` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:43.170767 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2443e1-8c18-460f-a8fa-e4c59f42d55a admin admin] RBAC: Authorizing `identity:delete_user(user_id=06fdc2590ce3471ab58d902957305606)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:43.173741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2443e1-8c18-460f-a8fa-e4c59f42d55a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:43.236209 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c605552-525e-41c7-b5f9-95e4128e4d6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '310f1a2f1f84472c9b611bbcf002da26', 'request_id': 'req-5c605552-525e-41c7-b5f9-95e4128e4d6c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.243892 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c605552-525e-41c7-b5f9-95e4128e4d6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '310f1a2f1f84472c9b611bbcf002da26', 'request_id': 'req-5c605552-525e-41c7-b5f9-95e4128e4d6c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.252102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c605552-525e-41c7-b5f9-95e4128e4d6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '310f1a2f1f84472c9b611bbcf002da26', 'request_id': 'req-5c605552-525e-41c7-b5f9-95e4128e4d6c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.265853 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fd2443e1-8c18-460f-a8fa-e4c59f42d55a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06fdc2590ce3471ab58d902957305606'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.272597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c605552-525e-41c7-b5f9-95e4128e4d6c admin admin] The token cache is being invalidate because project 310f1a2f1f84472c9b611bbcf002da26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:43.272597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c605552-525e-41c7-b5f9-95e4128e4d6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.273280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c605552-525e-41c7-b5f9-95e4128e4d6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.274995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 952/1937] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:43 2022] DELETE /identity/v3/projects/310f1a2f1f84472c9b611bbcf002da26 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:43.279670 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c605552-525e-41c7-b5f9-95e4128e4d6c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:43.282143 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c605552-525e-41c7-b5f9-95e4128e4d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:43.298769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fd2443e1-8c18-460f-a8fa-e4c59f42d55a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06fdc2590ce3471ab58d902957305606'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.300109 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fd2443e1-8c18-460f-a8fa-e4c59f42d55a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06fdc2590ce3471ab58d902957305606'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.301588 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fd2443e1-8c18-460f-a8fa-e4c59f42d55a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06fdc2590ce3471ab58d902957305606'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.324488 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fd2443e1-8c18-460f-a8fa-e4c59f42d55a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06fdc2590ce3471ab58d902957305606'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.331148 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:43.343511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 986/1938] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:42 2022] DELETE /identity/v3/users/06fdc2590ce3471ab58d902957305606 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:43.344308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2443e1-8c18-460f-a8fa-e4c59f42d55a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:43.346831 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fd2443e1-8c18-460f-a8fa-e4c59f42d55a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:43.386063 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c605552-525e-41c7-b5f9-95e4128e4d6c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:43.443073 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3bd9fd-7718-4cc3-af65-166d375d989d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:43.443740 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bd9fd-7718-4cc3-af65-166d375d989d service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:43.444024 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bd9fd-7718-4cc3-af65-166d375d989d service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:43.444302 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3bd9fd-7718-4cc3-af65-166d375d989d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:43.451461 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ae3bd9fd-7718-4cc3-af65-166d375d989d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:43.466462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:43.514449 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2443e1-8c18-460f-a8fa-e4c59f42d55a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:43.530121 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831514f-e818-4615-9742-7b0c328d6bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:43.532100 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831514f-e818-4615-9742-7b0c328d6bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:43.532221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831514f-e818-4615-9742-7b0c328d6bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:43.532536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831514f-e818-4615-9742-7b0c328d6bcd admin admin] PATH_INFO: `/v3/users/b7c5612789904891b1a318a0fca232ae` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:43.537082 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831514f-e818-4615-9742-7b0c328d6bcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7c5612789904891b1a318a0fca232ae)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:43.541656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831514f-e818-4615-9742-7b0c328d6bcd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:43.553581 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3bd9fd-7718-4cc3-af65-166d375d989d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:43.555660 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3bd9fd-7718-4cc3-af65-166d375d989d service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:43.569663 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 953/1939] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:43.571296 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf83d7f-d0d5-420d-8bde-15534aadf593 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:43.571795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf83d7f-d0d5-420d-8bde-15534aadf593 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:43.572544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf83d7f-d0d5-420d-8bde-15534aadf593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:43.599847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fcf83d7f-d0d5-420d-8bde-15534aadf593 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:43.601719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fcf83d7f-d0d5-420d-8bde-15534aadf593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:43.614700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 954/1940] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:43.614700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf83d7f-d0d5-420d-8bde-15534aadf593 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:43.614700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fcf83d7f-d0d5-420d-8bde-15534aadf593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:43.629881 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf83d7f-d0d5-420d-8bde-15534aadf593 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:43.635986 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393508f-f853-4583-95e7-4c959bce4874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:43.637033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393508f-f853-4583-95e7-4c959bce4874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:43.637606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393508f-f853-4583-95e7-4c959bce4874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:43.637718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393508f-f853-4583-95e7-4c959bce4874 admin admin] PATH_INFO: `/v3/projects/84d0ff2656794b17a70cbb92ac152297` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:43.639493 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393508f-f853-4583-95e7-4c959bce4874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84d0ff2656794b17a70cbb92ac152297)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:43.641493 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393508f-f853-4583-95e7-4c959bce4874 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:43.659382 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9831514f-e818-4615-9742-7b0c328d6bcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7c5612789904891b1a318a0fca232ae'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.673626 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9831514f-e818-4615-9742-7b0c328d6bcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7c5612789904891b1a318a0fca232ae'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.674425 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9831514f-e818-4615-9742-7b0c328d6bcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7c5612789904891b1a318a0fca232ae'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.675104 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9831514f-e818-4615-9742-7b0c328d6bcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7c5612789904891b1a318a0fca232ae'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.683058 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9831514f-e818-4615-9742-7b0c328d6bcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7c5612789904891b1a318a0fca232ae'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.701256 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 987/1941] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:43 2022] DELETE /identity/v3/users/b7c5612789904891b1a318a0fca232ae => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:43.701256 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831514f-e818-4615-9742-7b0c328d6bcd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:43.701497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9831514f-e818-4615-9742-7b0c328d6bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:43.704238 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1393508f-f853-4583-95e7-4c959bce4874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84d0ff2656794b17a70cbb92ac152297', 'request_id': 'req-1393508f-f853-4583-95e7-4c959bce4874'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.713389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1393508f-f853-4583-95e7-4c959bce4874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84d0ff2656794b17a70cbb92ac152297', 'request_id': 'req-1393508f-f853-4583-95e7-4c959bce4874'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.723943 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1393508f-f853-4583-95e7-4c959bce4874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84d0ff2656794b17a70cbb92ac152297', 'request_id': 'req-1393508f-f853-4583-95e7-4c959bce4874'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.747847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1393508f-f853-4583-95e7-4c959bce4874 admin admin] The token cache is being invalidate because project 84d0ff2656794b17a70cbb92ac152297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:43.748289 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1393508f-f853-4583-95e7-4c959bce4874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.749104 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1393508f-f853-4583-95e7-4c959bce4874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:43.751283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 955/1942] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:43 2022] DELETE /identity/v3/projects/84d0ff2656794b17a70cbb92ac152297 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:43.758073 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097a611-542a-44d9-b699-3f81330dbb1b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:43.758362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097a611-542a-44d9-b699-3f81330dbb1b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:43.759139 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097a611-542a-44d9-b699-3f81330dbb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:43.785749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7097a611-542a-44d9-b699-3f81330dbb1b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:43.787633 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7097a611-542a-44d9-b699-3f81330dbb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:43.841823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831514f-e818-4615-9742-7b0c328d6bcd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:43.845810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f98855-87a3-4ca5-b68a-f3fc64a7c39f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:43.846469 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f98855-87a3-4ca5-b68a-f3fc64a7c39f service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:43.846752 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f98855-87a3-4ca5-b68a-f3fc64a7c39f service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:43.847051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f98855-87a3-4ca5-b68a-f3fc64a7c39f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:43.848964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-12f98855-87a3-4ca5-b68a-f3fc64a7c39f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:43.879014 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:43.924031 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 956/1943] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:29:43.931305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f98855-87a3-4ca5-b68a-f3fc64a7c39f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:43.931764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea074bfe-66c3-43ac-8047-e52155e81e49 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:43.934409 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea074bfe-66c3-43ac-8047-e52155e81e49 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:43.934772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea074bfe-66c3-43ac-8047-e52155e81e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:43.940991 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f98855-87a3-4ca5-b68a-f3fc64a7c39f service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:43.967448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 988/1944] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:43.979277 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ea074bfe-66c3-43ac-8047-e52155e81e49 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:43.981059 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f98855-87a3-4ca5-b68a-f3fc64a7c39f service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:43.981459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ea074bfe-66c3-43ac-8047-e52155e81e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:43.983363 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-12f98855-87a3-4ca5-b68a-f3fc64a7c39f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:43.992742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 957/1945] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:43.996813 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea074bfe-66c3-43ac-8047-e52155e81e49 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:44.003223 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f98855-87a3-4ca5-b68a-f3fc64a7c39f service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:44.010879 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db77313-157b-4c08-b7c3-ace6a9cf50b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:44.011812 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db77313-157b-4c08-b7c3-ace6a9cf50b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:44.012233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db77313-157b-4c08-b7c3-ace6a9cf50b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:44.012532 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db77313-157b-4c08-b7c3-ace6a9cf50b3 admin admin] PATH_INFO: `/v3/projects/e2b9d43ca5a4485cbe4c85ab26e439f4` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:44.012640 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea074bfe-66c3-43ac-8047-e52155e81e49 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:44.014454 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db77313-157b-4c08-b7c3-ace6a9cf50b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2b9d43ca5a4485cbe4c85ab26e439f4)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:44.017928 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db77313-157b-4c08-b7c3-ace6a9cf50b3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:44.019136 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d10e76-a026-471d-9c32-141970e946d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:44.019592 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d10e76-a026-471d-9c32-141970e946d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:44.019994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d10e76-a026-471d-9c32-141970e946d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:44.020349 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d10e76-a026-471d-9c32-141970e946d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:44.021560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d10e76-a026-471d-9c32-141970e946d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:44.021976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d10e76-a026-471d-9c32-141970e946d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:44.023754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d10e76-a026-471d-9c32-141970e946d4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:44.043514 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 958/1946] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:44.049477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d10e76-a026-471d-9c32-141970e946d4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:44.059914 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:44.065783 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d10e76-a026-471d-9c32-141970e946d4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:44.073229 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23810ea1-faba-4fd1-9266-6a79bccbd3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:44.074111 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23810ea1-faba-4fd1-9266-6a79bccbd3f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:44.074562 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23810ea1-faba-4fd1-9266-6a79bccbd3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:44.074900 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23810ea1-faba-4fd1-9266-6a79bccbd3f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:44.075633 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23810ea1-faba-4fd1-9266-6a79bccbd3f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:44.078140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23810ea1-faba-4fd1-9266-6a79bccbd3f8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:44.096697 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5db77313-157b-4c08-b7c3-ace6a9cf50b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2b9d43ca5a4485cbe4c85ab26e439f4', 'request_id': 'req-5db77313-157b-4c08-b7c3-ace6a9cf50b3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:44.114420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 959/1947] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:44 2022] POST /identity/v3/projects => generated 407 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:44.119027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5db77313-157b-4c08-b7c3-ace6a9cf50b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2b9d43ca5a4485cbe4c85ab26e439f4', 'request_id': 'req-5db77313-157b-4c08-b7c3-ace6a9cf50b3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:44.119593 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23810ea1-faba-4fd1-9266-6a79bccbd3f8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:44.126453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5db77313-157b-4c08-b7c3-ace6a9cf50b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2b9d43ca5a4485cbe4c85ab26e439f4', 'request_id': 'req-5db77313-157b-4c08-b7c3-ace6a9cf50b3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:44.142290 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5db77313-157b-4c08-b7c3-ace6a9cf50b3 admin admin] The token cache is being invalidate because project e2b9d43ca5a4485cbe4c85ab26e439f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:44.142739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5db77313-157b-4c08-b7c3-ace6a9cf50b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:44.143646 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5db77313-157b-4c08-b7c3-ace6a9cf50b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:44.146775 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 989/1948] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:43 2022] DELETE /identity/v3/projects/e2b9d43ca5a4485cbe4c85ab26e439f4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:44.150420 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db77313-157b-4c08-b7c3-ace6a9cf50b3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:44.154104 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5db77313-157b-4c08-b7c3-ace6a9cf50b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:44.211044 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23810ea1-faba-4fd1-9266-6a79bccbd3f8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:44.216357 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd16bd-0525-42cf-98cc-8bafbef2e486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:44.217085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd16bd-0525-42cf-98cc-8bafbef2e486 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:44.217427 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd16bd-0525-42cf-98cc-8bafbef2e486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:44.217700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd16bd-0525-42cf-98cc-8bafbef2e486 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:44.218687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd16bd-0525-42cf-98cc-8bafbef2e486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:44.220673 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd16bd-0525-42cf-98cc-8bafbef2e486 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:44.249906 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 960/1949] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:44 2022] POST /identity/v3/users => generated 304 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:44.251118 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd16bd-0525-42cf-98cc-8bafbef2e486 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:44.254210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c5dd16bd-0525-42cf-98cc-8bafbef2e486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:44.255379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db77313-157b-4c08-b7c3-ace6a9cf50b3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:44.262236 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0218b78-f08a-4162-b26e-4f080725a2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:44.264414 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0218b78-f08a-4162-b26e-4f080725a2b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:44.264971 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0218b78-f08a-4162-b26e-4f080725a2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:44.265256 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0218b78-f08a-4162-b26e-4f080725a2b6 admin admin] PATH_INFO: `/v3/users/3c7c65c395e54952b332f390c6285a5f` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:44.267668 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0218b78-f08a-4162-b26e-4f080725a2b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c7c65c395e54952b332f390c6285a5f)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:44.268089 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd16bd-0525-42cf-98cc-8bafbef2e486 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:44.270243 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0218b78-f08a-4162-b26e-4f080725a2b6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:44.274951 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fce79-a79d-45c5-ae79-b2bd521a0223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:44.275639 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fce79-a79d-45c5-ae79-b2bd521a0223 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:44.276016 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fce79-a79d-45c5-ae79-b2bd521a0223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:44.276300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fce79-a79d-45c5-ae79-b2bd521a0223 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:44.277079 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fce79-a79d-45c5-ae79-b2bd521a0223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:44.278758 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fce79-a79d-45c5-ae79-b2bd521a0223 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:44.292539 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 961/1950] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:29:44.293736 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fce79-a79d-45c5-ae79-b2bd521a0223 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:44.305852 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fce79-a79d-45c5-ae79-b2bd521a0223 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:44.312975 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65351d42-963b-4040-8721-db9f690f50f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:44.313906 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65351d42-963b-4040-8721-db9f690f50f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:44.314275 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65351d42-963b-4040-8721-db9f690f50f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:44.314527 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65351d42-963b-4040-8721-db9f690f50f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:44.316091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65351d42-963b-4040-8721-db9f690f50f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:44.317103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65351d42-963b-4040-8721-db9f690f50f3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:44.334197 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 962/1951] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:44.338600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65351d42-963b-4040-8721-db9f690f50f3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:44.348796 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0218b78-f08a-4162-b26e-4f080725a2b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c7c65c395e54952b332f390c6285a5f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:44.351773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:44.354457 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65351d42-963b-4040-8721-db9f690f50f3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:44.365497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165b202-0c4b-4888-bd4f-d11e16362039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:44.367018 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165b202-0c4b-4888-bd4f-d11e16362039 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:44.367018 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165b202-0c4b-4888-bd4f-d11e16362039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:44.367511 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165b202-0c4b-4888-bd4f-d11e16362039 admin admin] PATH_INFO: `/v3/projects/222e4821207649d0b9b40e7f96b34873/users/653aa49b3a3e4171a1b285bb4c153c2e/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:44.382722 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0218b78-f08a-4162-b26e-4f080725a2b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c7c65c395e54952b332f390c6285a5f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:44.383830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0218b78-f08a-4162-b26e-4f080725a2b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c7c65c395e54952b332f390c6285a5f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:44.385143 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0218b78-f08a-4162-b26e-4f080725a2b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c7c65c395e54952b332f390c6285a5f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:44.397328 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165b202-0c4b-4888-bd4f-d11e16362039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=222e4821207649d0b9b40e7f96b34873, user_id=653aa49b3a3e4171a1b285bb4c153c2e, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:44.401410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165b202-0c4b-4888-bd4f-d11e16362039 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:44.401954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0218b78-f08a-4162-b26e-4f080725a2b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c7c65c395e54952b332f390c6285a5f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:44.424637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 963/1952] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:44 2022] PUT /identity/v3/projects/222e4821207649d0b9b40e7f96b34873/users/653aa49b3a3e4171a1b285bb4c153c2e/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:44.427427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 990/1953] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:44 2022] DELETE /identity/v3/users/3c7c65c395e54952b332f390c6285a5f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:44.433137 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250b6f5-73d3-4b9e-a950-fee0e3ce12f5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:44.433893 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250b6f5-73d3-4b9e-a950-fee0e3ce12f5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:44.433893 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250b6f5-73d3-4b9e-a950-fee0e3ce12f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:44.464425 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2250b6f5-73d3-4b9e-a950-fee0e3ce12f5 None None] MFA Rules not processed for user `653aa49b3a3e4171a1b285bb4c153c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:44.465927 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2250b6f5-73d3-4b9e-a950-fee0e3ce12f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:44.527432 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0218b78-f08a-4162-b26e-4f080725a2b6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:44.529224 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e0218b78-f08a-4162-b26e-4f080725a2b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:44.579273 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 964/1954] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:44 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:44.580204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250b6f5-73d3-4b9e-a950-fee0e3ce12f5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:44.582504 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2250b6f5-73d3-4b9e-a950-fee0e3ce12f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:44.607512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:44.667244 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0218b78-f08a-4162-b26e-4f080725a2b6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:44.667466 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250b6f5-73d3-4b9e-a950-fee0e3ce12f5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:44.672140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe99131-168f-4db3-94a8-d47cf59ade88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:44.672855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe99131-168f-4db3-94a8-d47cf59ade88 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:44.673140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe99131-168f-4db3-94a8-d47cf59ade88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:44.673415 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe99131-168f-4db3-94a8-d47cf59ade88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:44.674116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe99131-168f-4db3-94a8-d47cf59ade88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:44.676113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe99131-168f-4db3-94a8-d47cf59ade88 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:44.678479 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b646ce-4a7a-4837-abb1-1f88ed00c13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:44.680061 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b646ce-4a7a-4837-abb1-1f88ed00c13e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:44.682888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b646ce-4a7a-4837-abb1-1f88ed00c13e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:44.683300 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b646ce-4a7a-4837-abb1-1f88ed00c13e admin admin] PATH_INFO: `/v3/projects/58cc630e90f745fdade9c4740eba7d65` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:44.686350 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b646ce-4a7a-4837-abb1-1f88ed00c13e admin admin] RBAC: Authorizing `identity:delete_project(project_id=58cc630e90f745fdade9c4740eba7d65)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:44.690642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b646ce-4a7a-4837-abb1-1f88ed00c13e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:44.724409 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 965/1955] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:44 2022] POST /identity/v3/projects => generated 409 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:44.726428 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe99131-168f-4db3-94a8-d47cf59ade88 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:44.765025 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-91b646ce-4a7a-4837-abb1-1f88ed00c13e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58cc630e90f745fdade9c4740eba7d65', 'request_id': 'req-91b646ce-4a7a-4837-abb1-1f88ed00c13e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:44.773581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-91b646ce-4a7a-4837-abb1-1f88ed00c13e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58cc630e90f745fdade9c4740eba7d65', 'request_id': 'req-91b646ce-4a7a-4837-abb1-1f88ed00c13e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:44.786846 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-91b646ce-4a7a-4837-abb1-1f88ed00c13e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58cc630e90f745fdade9c4740eba7d65', 'request_id': 'req-91b646ce-4a7a-4837-abb1-1f88ed00c13e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:44.807630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-91b646ce-4a7a-4837-abb1-1f88ed00c13e admin admin] The token cache is being invalidate because project 58cc630e90f745fdade9c4740eba7d65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:44.807928 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-91b646ce-4a7a-4837-abb1-1f88ed00c13e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:44.809794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-91b646ce-4a7a-4837-abb1-1f88ed00c13e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:44.813122 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe99131-168f-4db3-94a8-d47cf59ade88 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:44.819278 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 991/1956] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:44 2022] DELETE /identity/v3/projects/58cc630e90f745fdade9c4740eba7d65 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:44.820713 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b646ce-4a7a-4837-abb1-1f88ed00c13e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:44.822562 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-91b646ce-4a7a-4837-abb1-1f88ed00c13e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:44.917160 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:44.922937 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418dadc1-8096-4a16-bf19-42e60df34f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:44.923690 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418dadc1-8096-4a16-bf19-42e60df34f31 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:44.924170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418dadc1-8096-4a16-bf19-42e60df34f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:44.924355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418dadc1-8096-4a16-bf19-42e60df34f31 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:44.925201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418dadc1-8096-4a16-bf19-42e60df34f31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:44.928074 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418dadc1-8096-4a16-bf19-42e60df34f31 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:44.930558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b646ce-4a7a-4837-abb1-1f88ed00c13e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:44.941915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6effc8-8700-41c6-9164-02ea369d6729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:44.943483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6effc8-8700-41c6-9164-02ea369d6729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:44.944724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6effc8-8700-41c6-9164-02ea369d6729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:44.945364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6effc8-8700-41c6-9164-02ea369d6729 admin admin] PATH_INFO: `/v3/users/2d30ef9771354fa6b3c70100207b83a4` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:44.946852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6effc8-8700-41c6-9164-02ea369d6729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d30ef9771354fa6b3c70100207b83a4)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:44.950618 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6effc8-8700-41c6-9164-02ea369d6729 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:44.974491 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 966/1957] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:44 2022] POST /identity/v3/users => generated 311 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:44.976887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418dadc1-8096-4a16-bf19-42e60df34f31 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:44.979759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-418dadc1-8096-4a16-bf19-42e60df34f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:44.996701 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418dadc1-8096-4a16-bf19-42e60df34f31 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:45.003477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c03905-520f-406e-be5e-8d2378e67b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:45.005175 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c03905-520f-406e-be5e-8d2378e67b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:45.006059 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c03905-520f-406e-be5e-8d2378e67b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:45.007185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c03905-520f-406e-be5e-8d2378e67b66 admin admin] PATH_INFO: `/v3/projects/4dc2b1805aa740ef87e8eaaac3815228` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:45.008860 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c03905-520f-406e-be5e-8d2378e67b66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dc2b1805aa740ef87e8eaaac3815228)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:45.011510 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c03905-520f-406e-be5e-8d2378e67b66 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:45.038045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0a6effc8-8700-41c6-9164-02ea369d6729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d30ef9771354fa6b3c70100207b83a4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.066055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0a6effc8-8700-41c6-9164-02ea369d6729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d30ef9771354fa6b3c70100207b83a4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.066817 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0a6effc8-8700-41c6-9164-02ea369d6729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d30ef9771354fa6b3c70100207b83a4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.067582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0a6effc8-8700-41c6-9164-02ea369d6729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d30ef9771354fa6b3c70100207b83a4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.085088 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:45.089631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0a6effc8-8700-41c6-9164-02ea369d6729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d30ef9771354fa6b3c70100207b83a4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.107299 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-27c03905-520f-406e-be5e-8d2378e67b66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dc2b1805aa740ef87e8eaaac3815228', 'request_id': 'req-27c03905-520f-406e-be5e-8d2378e67b66'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.118337 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-27c03905-520f-406e-be5e-8d2378e67b66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dc2b1805aa740ef87e8eaaac3815228', 'request_id': 'req-27c03905-520f-406e-be5e-8d2378e67b66'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.133860 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 992/1958] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:44 2022] DELETE /identity/v3/users/2d30ef9771354fa6b3c70100207b83a4 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:45.134150 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6effc8-8700-41c6-9164-02ea369d6729 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:45.137225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-27c03905-520f-406e-be5e-8d2378e67b66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dc2b1805aa740ef87e8eaaac3815228', 'request_id': 'req-27c03905-520f-406e-be5e-8d2378e67b66'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.141402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0a6effc8-8700-41c6-9164-02ea369d6729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:45.162497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-27c03905-520f-406e-be5e-8d2378e67b66 admin admin] The token cache is being invalidate because project 4dc2b1805aa740ef87e8eaaac3815228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:45.165237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-27c03905-520f-406e-be5e-8d2378e67b66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.166557 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-27c03905-520f-406e-be5e-8d2378e67b66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.172576 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 967/1959] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:44 2022] DELETE /identity/v3/projects/4dc2b1805aa740ef87e8eaaac3815228 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:45.173754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c03905-520f-406e-be5e-8d2378e67b66 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:45.176721 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-27c03905-520f-406e-be5e-8d2378e67b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:45.300193 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6effc8-8700-41c6-9164-02ea369d6729 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:45.305800 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aea62e-6274-4a80-9721-b1c7f395e8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:45.306492 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aea62e-6274-4a80-9721-b1c7f395e8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:45.306786 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aea62e-6274-4a80-9721-b1c7f395e8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:45.307071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aea62e-6274-4a80-9721-b1c7f395e8d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:45.307845 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aea62e-6274-4a80-9721-b1c7f395e8d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:45.320300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c03905-520f-406e-be5e-8d2378e67b66 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:45.328644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75eac4-704e-410a-b531-4d4074a3485d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:45.329786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75eac4-704e-410a-b531-4d4074a3485d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:45.330502 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75eac4-704e-410a-b531-4d4074a3485d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:45.332160 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75eac4-704e-410a-b531-4d4074a3485d admin admin] PATH_INFO: `/v3/projects/9b76defec9564410aacfede3c9420966` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:45.333329 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aea62e-6274-4a80-9721-b1c7f395e8d4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:45.334241 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b75eac4-704e-410a-b531-4d4074a3485d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b76defec9564410aacfede3c9420966)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:45.339113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b75eac4-704e-410a-b531-4d4074a3485d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:45.366060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 993/1960] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:45.368553 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38334112-01c3-4a4e-8c65-6d2470d60663 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:45.378198 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38334112-01c3-4a4e-8c65-6d2470d60663 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:45.381540 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38334112-01c3-4a4e-8c65-6d2470d60663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:45.415812 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1b75eac4-704e-410a-b531-4d4074a3485d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b76defec9564410aacfede3c9420966', 'request_id': 'req-1b75eac4-704e-410a-b531-4d4074a3485d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.424454 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1b75eac4-704e-410a-b531-4d4074a3485d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b76defec9564410aacfede3c9420966', 'request_id': 'req-1b75eac4-704e-410a-b531-4d4074a3485d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.431395 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-38334112-01c3-4a4e-8c65-6d2470d60663 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:45.432962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1b75eac4-704e-410a-b531-4d4074a3485d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b76defec9564410aacfede3c9420966', 'request_id': 'req-1b75eac4-704e-410a-b531-4d4074a3485d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.433126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-38334112-01c3-4a4e-8c65-6d2470d60663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:45.457694 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1b75eac4-704e-410a-b531-4d4074a3485d admin admin] The token cache is being invalidate because project 9b76defec9564410aacfede3c9420966 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:45.457905 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1b75eac4-704e-410a-b531-4d4074a3485d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.463547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1b75eac4-704e-410a-b531-4d4074a3485d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.465683 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 968/1961] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:45 2022] DELETE /identity/v3/projects/9b76defec9564410aacfede3c9420966 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:45.471809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75eac4-704e-410a-b531-4d4074a3485d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:45.476404 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1b75eac4-704e-410a-b531-4d4074a3485d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:45.626920 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 994/1962] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:45.626920 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38334112-01c3-4a4e-8c65-6d2470d60663 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:45.628109 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-38334112-01c3-4a4e-8c65-6d2470d60663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:45.633031 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:45.662470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38334112-01c3-4a4e-8c65-6d2470d60663 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:45.662708 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75eac4-704e-410a-b531-4d4074a3485d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:45.674416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc320d9-b0a6-439d-909b-8a2d4ae75cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:45.676817 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc320d9-b0a6-439d-909b-8a2d4ae75cd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:45.677434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc320d9-b0a6-439d-909b-8a2d4ae75cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:45.677777 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc320d9-b0a6-439d-909b-8a2d4ae75cd4 admin admin] PATH_INFO: `/v3/projects/770bdaf40ae74972ba29f8b847fa05ea/users/842cdc98597c41a7bec88a68e23f64e6/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:45.680419 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a25082-1450-4283-bf7f-6ef9ac6b0981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:45.681348 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a25082-1450-4283-bf7f-6ef9ac6b0981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:45.681637 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a25082-1450-4283-bf7f-6ef9ac6b0981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:45.681917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a25082-1450-4283-bf7f-6ef9ac6b0981 admin admin] PATH_INFO: `/v3/users/f612d09ec9c64f02aaacd514696642cd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:45.683152 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a25082-1450-4283-bf7f-6ef9ac6b0981 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f612d09ec9c64f02aaacd514696642cd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:45.685244 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a25082-1450-4283-bf7f-6ef9ac6b0981 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:45.691424 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:45.705565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc320d9-b0a6-439d-909b-8a2d4ae75cd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=770bdaf40ae74972ba29f8b847fa05ea, user_id=842cdc98597c41a7bec88a68e23f64e6, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:45.709894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc320d9-b0a6-439d-909b-8a2d4ae75cd4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:45.733113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 969/1963] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:45 2022] PUT /identity/v3/projects/770bdaf40ae74972ba29f8b847fa05ea/users/842cdc98597c41a7bec88a68e23f64e6/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:45.734751 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8717a-5111-46f6-96c6-ebde49c2d162 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:45.735023 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8717a-5111-46f6-96c6-ebde49c2d162 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:45.735396 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8717a-5111-46f6-96c6-ebde49c2d162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:45.760689 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-75e8717a-5111-46f6-96c6-ebde49c2d162 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:45.761981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-75e8717a-5111-46f6-96c6-ebde49c2d162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:45.775129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d6a25082-1450-4283-bf7f-6ef9ac6b0981 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f612d09ec9c64f02aaacd514696642cd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.813764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d6a25082-1450-4283-bf7f-6ef9ac6b0981 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f612d09ec9c64f02aaacd514696642cd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.814754 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d6a25082-1450-4283-bf7f-6ef9ac6b0981 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f612d09ec9c64f02aaacd514696642cd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.815479 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d6a25082-1450-4283-bf7f-6ef9ac6b0981 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f612d09ec9c64f02aaacd514696642cd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.826620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d6a25082-1450-4283-bf7f-6ef9ac6b0981 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f612d09ec9c64f02aaacd514696642cd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:45.844205 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 995/1964] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:45 2022] DELETE /identity/v3/users/f612d09ec9c64f02aaacd514696642cd => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:45.848433 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a25082-1450-4283-bf7f-6ef9ac6b0981 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:45.850134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d6a25082-1450-4283-bf7f-6ef9ac6b0981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:45.858553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 970/1965] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:45.867092 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8717a-5111-46f6-96c6-ebde49c2d162 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:45.882769 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8717a-5111-46f6-96c6-ebde49c2d162 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:45.952205 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a25082-1450-4283-bf7f-6ef9ac6b0981 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:45.962149 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1215104-55c4-4031-93e1-2987325eb5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:45.963919 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1215104-55c4-4031-93e1-2987325eb5d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:45.964105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1215104-55c4-4031-93e1-2987325eb5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:45.964407 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1215104-55c4-4031-93e1-2987325eb5d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:45.967966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1215104-55c4-4031-93e1-2987325eb5d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:45.969829 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1215104-55c4-4031-93e1-2987325eb5d9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:45.993281 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 996/1966] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:45.994908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1215104-55c4-4031-93e1-2987325eb5d9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:45.997178 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e1215104-55c4-4031-93e1-2987325eb5d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:46.003276 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff96885-ed3f-45fb-a568-ea253e4001c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:46.005408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff96885-ed3f-45fb-a568-ea253e4001c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:46.005732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff96885-ed3f-45fb-a568-ea253e4001c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:46.006143 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff96885-ed3f-45fb-a568-ea253e4001c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:46.006902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff96885-ed3f-45fb-a568-ea253e4001c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:46.007771 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff96885-ed3f-45fb-a568-ea253e4001c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:46.009944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff96885-ed3f-45fb-a568-ea253e4001c0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:46.056068 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 971/1967] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:46.056068 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff96885-ed3f-45fb-a568-ea253e4001c0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:46.059096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1215104-55c4-4031-93e1-2987325eb5d9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:46.068571 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d570be-deab-424a-b105-67eb1287485b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:46.069458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d570be-deab-424a-b105-67eb1287485b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:46.069742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d570be-deab-424a-b105-67eb1287485b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:46.070548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d570be-deab-424a-b105-67eb1287485b admin admin] PATH_INFO: `/v3/projects/fb6d650c5046405b99561e4ed0a8b4a1` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:46.072894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff96885-ed3f-45fb-a568-ea253e4001c0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:46.074207 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d570be-deab-424a-b105-67eb1287485b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb6d650c5046405b99561e4ed0a8b4a1)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:46.076138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d570be-deab-424a-b105-67eb1287485b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:46.087821 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a742284-9113-4e4e-b4ea-85fc152892fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:46.088544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a742284-9113-4e4e-b4ea-85fc152892fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:46.088835 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a742284-9113-4e4e-b4ea-85fc152892fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:46.089113 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a742284-9113-4e4e-b4ea-85fc152892fc admin admin] PATH_INFO: `/v3/projects/770bdaf40ae74972ba29f8b847fa05ea/users/842cdc98597c41a7bec88a68e23f64e6/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:46.091278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a742284-9113-4e4e-b4ea-85fc152892fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=770bdaf40ae74972ba29f8b847fa05ea, user_id=842cdc98597c41a7bec88a68e23f64e6, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:46.095125 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a742284-9113-4e4e-b4ea-85fc152892fc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:46.122180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 972/1968] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:46 2022] PUT /identity/v3/projects/770bdaf40ae74972ba29f8b847fa05ea/users/842cdc98597c41a7bec88a68e23f64e6/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:46.123252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a742284-9113-4e4e-b4ea-85fc152892fc admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:46.172404 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b0d570be-deab-424a-b105-67eb1287485b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb6d650c5046405b99561e4ed0a8b4a1', 'request_id': 'req-b0d570be-deab-424a-b105-67eb1287485b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:46.186690 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b0d570be-deab-424a-b105-67eb1287485b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb6d650c5046405b99561e4ed0a8b4a1', 'request_id': 'req-b0d570be-deab-424a-b105-67eb1287485b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:46.195040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b0d570be-deab-424a-b105-67eb1287485b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb6d650c5046405b99561e4ed0a8b4a1', 'request_id': 'req-b0d570be-deab-424a-b105-67eb1287485b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:46.209431 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:46.214244 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b0d570be-deab-424a-b105-67eb1287485b admin admin] The token cache is being invalidate because project fb6d650c5046405b99561e4ed0a8b4a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:46.214726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b0d570be-deab-424a-b105-67eb1287485b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:46.215537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b0d570be-deab-424a-b105-67eb1287485b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:46.215814 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a742284-9113-4e4e-b4ea-85fc152892fc admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:46.218229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 997/1969] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:45 2022] DELETE /identity/v3/projects/fb6d650c5046405b99561e4ed0a8b4a1 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:46.220931 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19465955-5b00-4bd7-ba97-261dfd821402 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:46.221260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19465955-5b00-4bd7-ba97-261dfd821402 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:46.221549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19465955-5b00-4bd7-ba97-261dfd821402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:46.234984 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:46.272144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-19465955-5b00-4bd7-ba97-261dfd821402 None None] MFA Rules not processed for user `842cdc98597c41a7bec88a68e23f64e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:46.277809 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-19465955-5b00-4bd7-ba97-261dfd821402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:46.314563 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa49f63-17ad-46b0-ac4c-c3667846ce1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:46.315395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa49f63-17ad-46b0-ac4c-c3667846ce1f admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:46.315783 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa49f63-17ad-46b0-ac4c-c3667846ce1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:46.316120 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa49f63-17ad-46b0-ac4c-c3667846ce1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:46.317447 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa49f63-17ad-46b0-ac4c-c3667846ce1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:46.319498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa49f63-17ad-46b0-ac4c-c3667846ce1f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:46.348377 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 973/1970] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:46 2022] POST /identity/v3/projects => generated 373 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:46.350283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990f8a7-b344-42c1-876d-0673e24b8281 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:46.350900 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990f8a7-b344-42c1-876d-0673e24b8281 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:46.351183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990f8a7-b344-42c1-876d-0673e24b8281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:46.371694 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d990f8a7-b344-42c1-876d-0673e24b8281 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:46.372953 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d990f8a7-b344-42c1-876d-0673e24b8281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:46.374022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 998/1971] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:46 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:46.374659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19465955-5b00-4bd7-ba97-261dfd821402 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:46.384239 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-19465955-5b00-4bd7-ba97-261dfd821402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:46.403701 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19465955-5b00-4bd7-ba97-261dfd821402 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:46.437571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 974/1972] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:46.438195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990f8a7-b344-42c1-876d-0673e24b8281 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:46.439472 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d990f8a7-b344-42c1-876d-0673e24b8281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:46.451760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990f8a7-b344-42c1-876d-0673e24b8281 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:46.457027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e934e0e-cd4e-46c7-9142-ceb4119a66f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:46.457921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e934e0e-cd4e-46c7-9142-ceb4119a66f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:46.458207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e934e0e-cd4e-46c7-9142-ceb4119a66f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:46.458675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e934e0e-cd4e-46c7-9142-ceb4119a66f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:46.459325 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e934e0e-cd4e-46c7-9142-ceb4119a66f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:46.461353 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e934e0e-cd4e-46c7-9142-ceb4119a66f6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:46.468478 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff01e4d-3ea3-468b-afbd-624229073ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:46.469457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff01e4d-3ea3-468b-afbd-624229073ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:46.470068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff01e4d-3ea3-468b-afbd-624229073ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:46.470147 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff01e4d-3ea3-468b-afbd-624229073ffc admin admin] PATH_INFO: `/v3/projects/f3c4986a06a74eb2b0dbdde79a35c482` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:46.471561 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff01e4d-3ea3-468b-afbd-624229073ffc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3c4986a06a74eb2b0dbdde79a35c482)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:46.473761 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff01e4d-3ea3-468b-afbd-624229073ffc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:46.484403 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 975/1973] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:46 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:46.485269 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e934e0e-cd4e-46c7-9142-ceb4119a66f6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:46.486795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3e934e0e-cd4e-46c7-9142-ceb4119a66f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:46.546220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0ff01e4d-3ea3-468b-afbd-624229073ffc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3c4986a06a74eb2b0dbdde79a35c482', 'request_id': 'req-0ff01e4d-3ea3-468b-afbd-624229073ffc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:46.556172 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0ff01e4d-3ea3-468b-afbd-624229073ffc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3c4986a06a74eb2b0dbdde79a35c482', 'request_id': 'req-0ff01e4d-3ea3-468b-afbd-624229073ffc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:46.563574 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e934e0e-cd4e-46c7-9142-ceb4119a66f6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:46.567065 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0ff01e4d-3ea3-468b-afbd-624229073ffc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3c4986a06a74eb2b0dbdde79a35c482', 'request_id': 'req-0ff01e4d-3ea3-468b-afbd-624229073ffc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:46.583329 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0ff01e4d-3ea3-468b-afbd-624229073ffc admin admin] The token cache is being invalidate because project f3c4986a06a74eb2b0dbdde79a35c482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:46.583644 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0ff01e4d-3ea3-468b-afbd-624229073ffc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:46.584620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0ff01e4d-3ea3-468b-afbd-624229073ffc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:46.586638 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 999/1974] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:46 2022] DELETE /identity/v3/projects/f3c4986a06a74eb2b0dbdde79a35c482 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:46.587365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee493ee-b492-4a0c-81d1-71d6c1e2a95a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:46.589055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee493ee-b492-4a0c-81d1-71d6c1e2a95a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:46.589055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee493ee-b492-4a0c-81d1-71d6c1e2a95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:46.598008 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:46.609161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fee493ee-b492-4a0c-81d1-71d6c1e2a95a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:46.610397 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fee493ee-b492-4a0c-81d1-71d6c1e2a95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:46.624265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:46.632023 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cbe1d-547f-4040-8940-e6ec31015543 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:46.632725 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cbe1d-547f-4040-8940-e6ec31015543 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:46.633016 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cbe1d-547f-4040-8940-e6ec31015543 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:46.633296 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cbe1d-547f-4040-8940-e6ec31015543 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:46.636180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-977cbe1d-547f-4040-8940-e6ec31015543 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:46.686619 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1000/1975] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:46.687466 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee493ee-b492-4a0c-81d1-71d6c1e2a95a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:46.690645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fee493ee-b492-4a0c-81d1-71d6c1e2a95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:46.703749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee493ee-b492-4a0c-81d1-71d6c1e2a95a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:46.712598 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01849058-6833-4848-8803-0ea30d1c1cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:46.713282 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01849058-6833-4848-8803-0ea30d1c1cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:46.713569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01849058-6833-4848-8803-0ea30d1c1cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:46.713852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01849058-6833-4848-8803-0ea30d1c1cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:46.714707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01849058-6833-4848-8803-0ea30d1c1cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:46.717233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01849058-6833-4848-8803-0ea30d1c1cc0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:46.732709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1001/1976] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:46.734168 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dd673-7b51-47c1-855f-3efbd755b093 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:46.734636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dd673-7b51-47c1-855f-3efbd755b093 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:46.734919 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dd673-7b51-47c1-855f-3efbd755b093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:46.753158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977cbe1d-547f-4040-8940-e6ec31015543 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:46.755827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977cbe1d-547f-4040-8940-e6ec31015543 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:46.765335 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d28dd673-7b51-47c1-855f-3efbd755b093 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:46.768185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 976/1977] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:46 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:46.770186 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cbe1d-547f-4040-8940-e6ec31015543 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:46.780778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d28dd673-7b51-47c1-855f-3efbd755b093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:46.782149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cbe1d-547f-4040-8940-e6ec31015543 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:46.788221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecee05-4271-4664-aed3-cdc419a58696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:46.789306 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecee05-4271-4664-aed3-cdc419a58696 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:46.789709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecee05-4271-4664-aed3-cdc419a58696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:46.790079 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecee05-4271-4664-aed3-cdc419a58696 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:46.791042 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cecee05-4271-4664-aed3-cdc419a58696 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:46.791531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cecee05-4271-4664-aed3-cdc419a58696 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:46.793394 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cecee05-4271-4664-aed3-cdc419a58696 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:46.812687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 977/1978] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:46.813833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecee05-4271-4664-aed3-cdc419a58696 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:46.814274 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1002/1979] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:46.822137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dd673-7b51-47c1-855f-3efbd755b093 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:46.842285 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dd673-7b51-47c1-855f-3efbd755b093 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:46.846214 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecee05-4271-4664-aed3-cdc419a58696 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:46.853162 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddca9a4-0ecd-4948-a3d8-899f107a49da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:46.855155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddca9a4-0ecd-4948-a3d8-899f107a49da admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:46.855756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddca9a4-0ecd-4948-a3d8-899f107a49da admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:46.856202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddca9a4-0ecd-4948-a3d8-899f107a49da admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:46.857276 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef127839-3b0c-4e50-af81-2d7e74fa0e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:46.857843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddca9a4-0ecd-4948-a3d8-899f107a49da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:46.858025 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef127839-3b0c-4e50-af81-2d7e74fa0e1d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:46.858300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef127839-3b0c-4e50-af81-2d7e74fa0e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:46.858577 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef127839-3b0c-4e50-af81-2d7e74fa0e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:46.859309 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef127839-3b0c-4e50-af81-2d7e74fa0e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:46.861650 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddca9a4-0ecd-4948-a3d8-899f107a49da admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:46.864233 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef127839-3b0c-4e50-af81-2d7e74fa0e1d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:46.890158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 978/1980] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:46.892982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5b6aa-c54b-4eb1-b91f-05bdc08483f5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:46.897544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5b6aa-c54b-4eb1-b91f-05bdc08483f5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:46.898500 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5b6aa-c54b-4eb1-b91f-05bdc08483f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:46.929511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1003/1981] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:46 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:46.930500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddca9a4-0ecd-4948-a3d8-899f107a49da admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:46.944268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cce5b6aa-c54b-4eb1-b91f-05bdc08483f5 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:46.946061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cce5b6aa-c54b-4eb1-b91f-05bdc08483f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:47.035692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddca9a4-0ecd-4948-a3d8-899f107a49da admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:47.042485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f54c00-80be-4060-ac9a-6e920ce80a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:47.043699 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f54c00-80be-4060-ac9a-6e920ce80a7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:47.043988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f54c00-80be-4060-ac9a-6e920ce80a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:47.044401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f54c00-80be-4060-ac9a-6e920ce80a7d admin admin] PATH_INFO: `/v3/projects/9698a0142f494095a93a4e804437bc32/users/bc233897770f47e5bd40fb03afabe49c/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:47.086681 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f54c00-80be-4060-ac9a-6e920ce80a7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9698a0142f494095a93a4e804437bc32, user_id=bc233897770f47e5bd40fb03afabe49c, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:47.089663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f54c00-80be-4060-ac9a-6e920ce80a7d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:47.102264 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 979/1982] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:47.103493 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5b6aa-c54b-4eb1-b91f-05bdc08483f5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:47.120987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1004/1983] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:46 2022] PUT /identity/v3/projects/9698a0142f494095a93a4e804437bc32/users/bc233897770f47e5bd40fb03afabe49c/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:47.122218 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f54c00-80be-4060-ac9a-6e920ce80a7d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:47.133793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-45f54c00-80be-4060-ac9a-6e920ce80a7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:47.139639 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5b6aa-c54b-4eb1-b91f-05bdc08483f5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:47.294030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:47.297006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f54c00-80be-4060-ac9a-6e920ce80a7d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:47.306497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f54be5-3874-4622-8126-0209e1e3edb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:47.307258 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f54be5-3874-4622-8126-0209e1e3edb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:47.307527 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f54be5-3874-4622-8126-0209e1e3edb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:47.308239 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f54be5-3874-4622-8126-0209e1e3edb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:47.309045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f54be5-3874-4622-8126-0209e1e3edb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:47.311058 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f54be5-3874-4622-8126-0209e1e3edb0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:47.311960 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532628b9-eb5e-451a-b6d8-4d3dd52f7fd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:47.314592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532628b9-eb5e-451a-b6d8-4d3dd52f7fd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:47.314966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532628b9-eb5e-451a-b6d8-4d3dd52f7fd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:47.315319 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532628b9-eb5e-451a-b6d8-4d3dd52f7fd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:47.318595 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-532628b9-eb5e-451a-b6d8-4d3dd52f7fd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:47.348861 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 980/1984] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:47 2022] POST /identity/v3/users => generated 294 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:47.350526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f54be5-3874-4622-8126-0209e1e3edb0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:47.365062 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f54be5-3874-4622-8126-0209e1e3edb0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:47.375676 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005158a-c6b3-424d-87d2-9988c0f203b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:47.375676 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005158a-c6b3-424d-87d2-9988c0f203b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:47.375676 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005158a-c6b3-424d-87d2-9988c0f203b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:47.375676 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005158a-c6b3-424d-87d2-9988c0f203b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:47.376710 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d005158a-c6b3-424d-87d2-9988c0f203b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:47.377087 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d005158a-c6b3-424d-87d2-9988c0f203b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:47.379714 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d005158a-c6b3-424d-87d2-9988c0f203b5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:47.386852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:47.395210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 981/1985] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:47.396864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cac1a9-de68-440c-aa62-70efae920c64 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:47.397165 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cac1a9-de68-440c-aa62-70efae920c64 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:47.397506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cac1a9-de68-440c-aa62-70efae920c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:47.418642 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-28cac1a9-de68-440c-aa62-70efae920c64 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:47.420533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-28cac1a9-de68-440c-aa62-70efae920c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:47.434354 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 982/1986] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:47.435373 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cac1a9-de68-440c-aa62-70efae920c64 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:47.448882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cac1a9-de68-440c-aa62-70efae920c64 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:47.457332 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48034d83-32dd-4c7b-a2d5-fcf056810ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:47.459497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48034d83-32dd-4c7b-a2d5-fcf056810ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:47.459497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48034d83-32dd-4c7b-a2d5-fcf056810ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:47.459497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48034d83-32dd-4c7b-a2d5-fcf056810ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:47.470608 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48034d83-32dd-4c7b-a2d5-fcf056810ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:47.472458 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48034d83-32dd-4c7b-a2d5-fcf056810ad5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:47.493714 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 983/1987] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:47.494902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48034d83-32dd-4c7b-a2d5-fcf056810ad5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:47.514993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48034d83-32dd-4c7b-a2d5-fcf056810ad5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:47.517424 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532628b9-eb5e-451a-b6d8-4d3dd52f7fd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:47.519202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532628b9-eb5e-451a-b6d8-4d3dd52f7fd3 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:47.531132 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564491e-3090-4a1c-8427-51c272ee16cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:47.531822 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564491e-3090-4a1c-8427-51c272ee16cd admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:47.532111 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564491e-3090-4a1c-8427-51c272ee16cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:47.532606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564491e-3090-4a1c-8427-51c272ee16cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:47.533320 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564491e-3090-4a1c-8427-51c272ee16cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:47.546730 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564491e-3090-4a1c-8427-51c272ee16cd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:47.550693 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1005/1988] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:47 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:47.552830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532628b9-eb5e-451a-b6d8-4d3dd52f7fd3 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:47.557785 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-532628b9-eb5e-451a-b6d8-4d3dd52f7fd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:47.613791 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 984/1989] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:47 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:47.614994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564491e-3090-4a1c-8427-51c272ee16cd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:47.668846 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532628b9-eb5e-451a-b6d8-4d3dd52f7fd3 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:47.726965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564491e-3090-4a1c-8427-51c272ee16cd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:47.745129 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df35ef-522b-4183-9455-a5f8a1eba9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:47.749494 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df35ef-522b-4183-9455-a5f8a1eba9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:47.749805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df35ef-522b-4183-9455-a5f8a1eba9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:47.750088 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df35ef-522b-4183-9455-a5f8a1eba9b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:47.750850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df35ef-522b-4183-9455-a5f8a1eba9b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:47.756258 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df35ef-522b-4183-9455-a5f8a1eba9b1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:47.763004 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda3a2e-834c-47dd-b04b-fec1183a2ac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:47.763677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda3a2e-834c-47dd-b04b-fec1183a2ac1 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:47.763968 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda3a2e-834c-47dd-b04b-fec1183a2ac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:47.764248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda3a2e-834c-47dd-b04b-fec1183a2ac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:47.786978 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 985/1990] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:47.787689 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df35ef-522b-4183-9455-a5f8a1eba9b1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:47.795920 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda3a2e-834c-47dd-b04b-fec1183a2ac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:47.797417 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda3a2e-834c-47dd-b04b-fec1183a2ac1 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:47.823163 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df35ef-522b-4183-9455-a5f8a1eba9b1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:47.836493 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1006/1991] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:47.838379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda3a2e-834c-47dd-b04b-fec1183a2ac1 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:47.840146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14599daf-fad6-4b2a-9a49-593ad5dfdcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:47.841913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14599daf-fad6-4b2a-9a49-593ad5dfdcb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:47.842210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14599daf-fad6-4b2a-9a49-593ad5dfdcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:47.842540 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14599daf-fad6-4b2a-9a49-593ad5dfdcb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:47.849698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14599daf-fad6-4b2a-9a49-593ad5dfdcb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:47.859127 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14599daf-fad6-4b2a-9a49-593ad5dfdcb2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:47.880445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda3a2e-834c-47dd-b04b-fec1183a2ac1 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:47.896488 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701e701-c636-4449-a396-2f17eb1fc402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:47.899101 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701e701-c636-4449-a396-2f17eb1fc402 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:47.899595 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701e701-c636-4449-a396-2f17eb1fc402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:47.899724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701e701-c636-4449-a396-2f17eb1fc402 admin admin] PATH_INFO: `/v3/projects/5fab27e1dbca4b2194fc9e917868472f/users/993d21698b984195bfe32a20fc4f0694/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:47.920220 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 986/1992] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:47 2022] POST /identity/v3/users => generated 286 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:47.924408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14599daf-fad6-4b2a-9a49-593ad5dfdcb2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:47.937792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701e701-c636-4449-a396-2f17eb1fc402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fab27e1dbca4b2194fc9e917868472f, user_id=993d21698b984195bfe32a20fc4f0694, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:47.942490 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14599daf-fad6-4b2a-9a49-593ad5dfdcb2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:47.944008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701e701-c636-4449-a396-2f17eb1fc402 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:47.953404 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfa3bb-bd44-4b15-8733-ad668e1eae89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:47.954369 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfa3bb-bd44-4b15-8733-ad668e1eae89 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:47.955858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfa3bb-bd44-4b15-8733-ad668e1eae89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:47.955858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfa3bb-bd44-4b15-8733-ad668e1eae89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:47.955858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdfa3bb-bd44-4b15-8733-ad668e1eae89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:47.958549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdfa3bb-bd44-4b15-8733-ad668e1eae89 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:47.976316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1007/1993] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:47 2022] PUT /identity/v3/projects/5fab27e1dbca4b2194fc9e917868472f/users/993d21698b984195bfe32a20fc4f0694/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:47.989325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c5adf-f294-4fca-b4a8-0bdea37ed9c6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:47.989618 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c5adf-f294-4fca-b4a8-0bdea37ed9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:47.989953 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c5adf-f294-4fca-b4a8-0bdea37ed9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:47.997931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 987/1994] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:48.003013 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfa3bb-bd44-4b15-8733-ad668e1eae89 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:48.029594 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfa3bb-bd44-4b15-8733-ad668e1eae89 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:48.056142 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-835c5adf-f294-4fca-b4a8-0bdea37ed9c6 None None] MFA Rules not processed for user `993d21698b984195bfe32a20fc4f0694`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:48.061013 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-835c5adf-f294-4fca-b4a8-0bdea37ed9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:48.199351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:48.220409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:48.225818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed4887-f3fa-4cc1-849a-b4e569d06cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:48.226761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed4887-f3fa-4cc1-849a-b4e569d06cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:48.227102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed4887-f3fa-4cc1-849a-b4e569d06cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:48.227477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed4887-f3fa-4cc1-849a-b4e569d06cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:48.228321 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed4887-f3fa-4cc1-849a-b4e569d06cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:48.230705 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed4887-f3fa-4cc1-849a-b4e569d06cf6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:48.263491 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 988/1995] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:48.264076 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1008/1996] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:29:48.268156 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c5adf-f294-4fca-b4a8-0bdea37ed9c6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:48.268469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed4887-f3fa-4cc1-849a-b4e569d06cf6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:48.283080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed4887-f3fa-4cc1-849a-b4e569d06cf6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:48.289088 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2519a-f146-435a-a8a8-9c26fbc9281f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:48.290295 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2519a-f146-435a-a8a8-9c26fbc9281f admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:48.290652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2519a-f146-435a-a8a8-9c26fbc9281f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:48.291524 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2519a-f146-435a-a8a8-9c26fbc9281f admin admin] PATH_INFO: `/v3/projects/9d62dc4cb3c04b298f2603948e13e135/users/476abb1c707f4d838559232d7e7d5986/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:48.305850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c5adf-f294-4fca-b4a8-0bdea37ed9c6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:48.309926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2519a-f146-435a-a8a8-9c26fbc9281f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d62dc4cb3c04b298f2603948e13e135, user_id=476abb1c707f4d838559232d7e7d5986, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:48.311389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfae455-48ed-471e-9e91-6364408f4bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:48.312069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfae455-48ed-471e-9e91-6364408f4bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:48.312357 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfae455-48ed-471e-9e91-6364408f4bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:48.312680 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfae455-48ed-471e-9e91-6364408f4bb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:48.313430 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2519a-f146-435a-a8a8-9c26fbc9281f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:48.313554 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfae455-48ed-471e-9e91-6364408f4bb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:48.316222 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfae455-48ed-471e-9e91-6364408f4bb1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:48.330079 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 989/1997] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:48 2022] PUT /identity/v3/projects/9d62dc4cb3c04b298f2603948e13e135/users/476abb1c707f4d838559232d7e7d5986/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:48.341206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c7392-e405-4274-a204-9d10a50bf629 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:48.341504 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c7392-e405-4274-a204-9d10a50bf629 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:48.342384 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c7392-e405-4274-a204-9d10a50bf629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:48.365381 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1009/1998] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:48 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:29:48.369069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfae455-48ed-471e-9e91-6364408f4bb1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:48.394098 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-623c7392-e405-4274-a204-9d10a50bf629 None None] MFA Rules not processed for user `476abb1c707f4d838559232d7e7d5986`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:48.396552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-623c7392-e405-4274-a204-9d10a50bf629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:48.509959 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfae455-48ed-471e-9e91-6364408f4bb1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:48.528199 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfbf6b-b3a3-4134-b19d-213dd88146f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:48.529560 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfbf6b-b3a3-4134-b19d-213dd88146f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:48.530011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfbf6b-b3a3-4134-b19d-213dd88146f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:48.530469 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfbf6b-b3a3-4134-b19d-213dd88146f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:48.531538 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfbf6b-b3a3-4134-b19d-213dd88146f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:48.539980 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfbf6b-b3a3-4134-b19d-213dd88146f7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:48.578760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 990/1999] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:48.584740 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c7392-e405-4274-a204-9d10a50bf629 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:48.588468 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1010/2000] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:48 2022] POST /identity/v3/users => generated 300 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:29:48.589325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfbf6b-b3a3-4134-b19d-213dd88146f7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:48.590343 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-623c7392-e405-4274-a204-9d10a50bf629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:48.605724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfbf6b-b3a3-4134-b19d-213dd88146f7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:48.622796 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885c2f1-6fae-4aef-81ff-ea816fb703ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:48.623727 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885c2f1-6fae-4aef-81ff-ea816fb703ed admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:48.623886 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885c2f1-6fae-4aef-81ff-ea816fb703ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:48.624059 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885c2f1-6fae-4aef-81ff-ea816fb703ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:48.624959 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a885c2f1-6fae-4aef-81ff-ea816fb703ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:48.626917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a885c2f1-6fae-4aef-81ff-ea816fb703ed admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:48.644946 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1011/2001] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:48.648805 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885c2f1-6fae-4aef-81ff-ea816fb703ed admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:48.664786 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885c2f1-6fae-4aef-81ff-ea816fb703ed admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:48.670487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ab973-4e62-4908-8ac4-1f453835a0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:48.671810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ab973-4e62-4908-8ac4-1f453835a0a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:48.671958 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ab973-4e62-4908-8ac4-1f453835a0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:48.672245 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ab973-4e62-4908-8ac4-1f453835a0a2 admin admin] PATH_INFO: `/v3/projects/a361c0842f6c4357815ade2afcd1d377/users/7f0d36408aac4ee9882aeb99564afe15/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:48.705890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c7392-e405-4274-a204-9d10a50bf629 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:48.712065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdacd5-51fb-4a61-a146-7fc931fe7b48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:48.713126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdacd5-51fb-4a61-a146-7fc931fe7b48 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:48.713675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdacd5-51fb-4a61-a146-7fc931fe7b48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:48.714115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdacd5-51fb-4a61-a146-7fc931fe7b48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:48.719331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ab973-4e62-4908-8ac4-1f453835a0a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a361c0842f6c4357815ade2afcd1d377, user_id=7f0d36408aac4ee9882aeb99564afe15, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:48.724126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ab973-4e62-4908-8ac4-1f453835a0a2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:48.726270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdacd5-51fb-4a61-a146-7fc931fe7b48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:48.727982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdacd5-51fb-4a61-a146-7fc931fe7b48 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:48.739682 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 991/2002] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:48.756723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdacd5-51fb-4a61-a146-7fc931fe7b48 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:48.756827 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1012/2003] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:48 2022] PUT /identity/v3/projects/a361c0842f6c4357815ade2afcd1d377/users/7f0d36408aac4ee9882aeb99564afe15/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:48.762977 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ab973-4e62-4908-8ac4-1f453835a0a2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:48.794692 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdacd5-51fb-4a61-a146-7fc931fe7b48 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:48.912924 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ce054-f521-458c-9043-7010d1ef94cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:48.913787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ce054-f521-458c-9043-7010d1ef94cd admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:48.914267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ce054-f521-458c-9043-7010d1ef94cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:48.914674 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ce054-f521-458c-9043-7010d1ef94cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:48.915523 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ce054-f521-458c-9043-7010d1ef94cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:48.917317 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ce054-f521-458c-9043-7010d1ef94cd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:48.921712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:48.940842 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 992/2004] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:48.946664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ce054-f521-458c-9043-7010d1ef94cd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:48.979463 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ce054-f521-458c-9043-7010d1ef94cd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:48.986858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ab973-4e62-4908-8ac4-1f453835a0a2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:48.999833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7acb696-1ba8-4bec-afe7-aa7b79f7f8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:49.000910 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acb696-1ba8-4bec-afe7-aa7b79f7f8ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:49.001570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acb696-1ba8-4bec-afe7-aa7b79f7f8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:49.001570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acb696-1ba8-4bec-afe7-aa7b79f7f8ba admin admin] PATH_INFO: `/v3/users/476abb1c707f4d838559232d7e7d5986` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:49.004238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7acb696-1ba8-4bec-afe7-aa7b79f7f8ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=476abb1c707f4d838559232d7e7d5986)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:49.006244 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7acb696-1ba8-4bec-afe7-aa7b79f7f8ba admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:49.019562 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0cd54-0741-491c-8ad0-b501aea22e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:49.022979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0cd54-0741-491c-8ad0-b501aea22e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:49.023315 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0cd54-0741-491c-8ad0-b501aea22e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:49.023609 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0cd54-0741-491c-8ad0-b501aea22e2e admin admin] PATH_INFO: `/v3/projects/a361c0842f6c4357815ade2afcd1d377/users/7f0d36408aac4ee9882aeb99564afe15/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:49.026123 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0cd54-0741-491c-8ad0-b501aea22e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a361c0842f6c4357815ade2afcd1d377, user_id=7f0d36408aac4ee9882aeb99564afe15, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:49.030949 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0cd54-0741-491c-8ad0-b501aea22e2e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:49.033081 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:49.064613 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 993/2005] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:48 2022] PUT /identity/v3/projects/a361c0842f6c4357815ade2afcd1d377/users/7f0d36408aac4ee9882aeb99564afe15/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:49.069294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09020d-19e3-4945-a10d-012c32322fbe None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:49.069596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09020d-19e3-4945-a10d-012c32322fbe None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:49.069890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09020d-19e3-4945-a10d-012c32322fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:49.119143 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ff09020d-19e3-4945-a10d-012c32322fbe None None] MFA Rules not processed for user `7f0d36408aac4ee9882aeb99564afe15`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:49.120390 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ff09020d-19e3-4945-a10d-012c32322fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:49.163228 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f7acb696-1ba8-4bec-afe7-aa7b79f7f8ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '476abb1c707f4d838559232d7e7d5986'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:49.219020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f7acb696-1ba8-4bec-afe7-aa7b79f7f8ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '476abb1c707f4d838559232d7e7d5986'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:49.220222 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f7acb696-1ba8-4bec-afe7-aa7b79f7f8ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '476abb1c707f4d838559232d7e7d5986'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:49.221087 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f7acb696-1ba8-4bec-afe7-aa7b79f7f8ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '476abb1c707f4d838559232d7e7d5986'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:49.243625 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f7acb696-1ba8-4bec-afe7-aa7b79f7f8ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '476abb1c707f4d838559232d7e7d5986'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:49.266257 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1013/2006] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:48 2022] DELETE /identity/v3/users/476abb1c707f4d838559232d7e7d5986 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:49.271305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f39424-cce0-4fcf-8dbb-96aabc2e8302 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:49.271661 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f39424-cce0-4fcf-8dbb-96aabc2e8302 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:49.271887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f39424-cce0-4fcf-8dbb-96aabc2e8302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:49.298261 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a1f39424-cce0-4fcf-8dbb-96aabc2e8302 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:49.299560 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a1f39424-cce0-4fcf-8dbb-96aabc2e8302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:49.320787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 994/2007] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:49.332367 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09020d-19e3-4945-a10d-012c32322fbe None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:49.332367 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ff09020d-19e3-4945-a10d-012c32322fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:49.442627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1014/2008] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:29:49.458745 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f39424-cce0-4fcf-8dbb-96aabc2e8302 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:49.459056 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09020d-19e3-4945-a10d-012c32322fbe None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:49.461175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a1f39424-cce0-4fcf-8dbb-96aabc2e8302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:49.543354 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68580de8-5bde-4427-8a77-33f990d921b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:49.544258 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68580de8-5bde-4427-8a77-33f990d921b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:49.544720 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68580de8-5bde-4427-8a77-33f990d921b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:49.545056 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68580de8-5bde-4427-8a77-33f990d921b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:49.644666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:49.657117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f39424-cce0-4fcf-8dbb-96aabc2e8302 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:49.668121 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b5267-cb2d-4538-bdcd-a9a86f04b04a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:49.669171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b5267-cb2d-4538-bdcd-a9a86f04b04a service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:49.670533 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b5267-cb2d-4538-bdcd-a9a86f04b04a service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:49.670672 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b5267-cb2d-4538-bdcd-a9a86f04b04a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:49.692106 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68580de8-5bde-4427-8a77-33f990d921b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:49.693018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b5267-cb2d-4538-bdcd-a9a86f04b04a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:49.695832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b5267-cb2d-4538-bdcd-a9a86f04b04a service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:49.696381 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68580de8-5bde-4427-8a77-33f990d921b8 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:49.724122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1015/2009] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:29:49.726898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 995/2010] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:49.727052 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88227527-3605-465f-beba-eca53f675f7e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:49.727859 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88227527-3605-465f-beba-eca53f675f7e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:49.728589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88227527-3605-465f-beba-eca53f675f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:49.771525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-88227527-3605-465f-beba-eca53f675f7e None None] MFA Rules not processed for user `bc233897770f47e5bd40fb03afabe49c`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:49.775045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-88227527-3605-465f-beba-eca53f675f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:49.852300 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:49.961729 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1016/2011] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:49.972831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68580de8-5bde-4427-8a77-33f990d921b8 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:49.974593 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-68580de8-5bde-4427-8a77-33f990d921b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:49.994071 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68580de8-5bde-4427-8a77-33f990d921b8 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:50.006875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42917d-2866-42d2-a5e1-90e902c0bb67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:50.009471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42917d-2866-42d2-a5e1-90e902c0bb67 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:50.009471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42917d-2866-42d2-a5e1-90e902c0bb67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:50.009765 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42917d-2866-42d2-a5e1-90e902c0bb67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:50.022409 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42917d-2866-42d2-a5e1-90e902c0bb67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:50.024253 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42917d-2866-42d2-a5e1-90e902c0bb67 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:50.036642 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 996/2012] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:50.448908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88227527-3605-465f-beba-eca53f675f7e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:50.450447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-88227527-3605-465f-beba-eca53f675f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:50.489674 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88227527-3605-465f-beba-eca53f675f7e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:50.506122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a042cd-48c6-4397-9116-4a439a239532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:50.506531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a042cd-48c6-4397-9116-4a439a239532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:50.511031 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a042cd-48c6-4397-9116-4a439a239532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:50.511327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a042cd-48c6-4397-9116-4a439a239532 admin admin] PATH_INFO: `/v3/projects/9d62dc4cb3c04b298f2603948e13e135` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:50.513844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a042cd-48c6-4397-9116-4a439a239532 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d62dc4cb3c04b298f2603948e13e135)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:50.516401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a042cd-48c6-4397-9116-4a439a239532 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:50.601475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-57a042cd-48c6-4397-9116-4a439a239532 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d62dc4cb3c04b298f2603948e13e135', 'request_id': 'req-57a042cd-48c6-4397-9116-4a439a239532'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:50.610679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-57a042cd-48c6-4397-9116-4a439a239532 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d62dc4cb3c04b298f2603948e13e135', 'request_id': 'req-57a042cd-48c6-4397-9116-4a439a239532'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:50.620942 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-57a042cd-48c6-4397-9116-4a439a239532 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d62dc4cb3c04b298f2603948e13e135', 'request_id': 'req-57a042cd-48c6-4397-9116-4a439a239532'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:50.640870 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-57a042cd-48c6-4397-9116-4a439a239532 admin admin] The token cache is being invalidate because project 9d62dc4cb3c04b298f2603948e13e135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:50.641317 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-57a042cd-48c6-4397-9116-4a439a239532 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:50.642008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-57a042cd-48c6-4397-9116-4a439a239532 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:50.644535 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1017/2013] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:50 2022] DELETE /identity/v3/projects/9d62dc4cb3c04b298f2603948e13e135 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:50.650941 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9198b7-4801-43ae-8af7-b8a6b6ad2f68 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:50.651819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9198b7-4801-43ae-8af7-b8a6b6ad2f68 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:50.652105 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9198b7-4801-43ae-8af7-b8a6b6ad2f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:50.674810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1f9198b7-4801-43ae-8af7-b8a6b6ad2f68 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:50.676639 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1f9198b7-4801-43ae-8af7-b8a6b6ad2f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:50.716740 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a042cd-48c6-4397-9116-4a439a239532 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:50.718323 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-57a042cd-48c6-4397-9116-4a439a239532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:50.748663 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 997/2014] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:50.756386 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b4d17-fe26-47ac-9ea0-341b66da0597 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:50.756639 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b4d17-fe26-47ac-9ea0-341b66da0597 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:50.756925 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b4d17-fe26-47ac-9ea0-341b66da0597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:50.776618 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-224b4d17-fe26-47ac-9ea0-341b66da0597 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:50.777848 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-224b4d17-fe26-47ac-9ea0-341b66da0597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:50.789970 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 998/2015] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:50.791072 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b4d17-fe26-47ac-9ea0-341b66da0597 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:50.806547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b4d17-fe26-47ac-9ea0-341b66da0597 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:50.808487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a042cd-48c6-4397-9116-4a439a239532 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:50.811422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8e770-867f-4de2-bbd1-c01db4c66751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:50.812087 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8e770-867f-4de2-bbd1-c01db4c66751 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:50.812368 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8e770-867f-4de2-bbd1-c01db4c66751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:50.812691 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8e770-867f-4de2-bbd1-c01db4c66751 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:50.813546 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8e770-867f-4de2-bbd1-c01db4c66751 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:50.813856 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8e770-867f-4de2-bbd1-c01db4c66751 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:50.815048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ddbc4-aae3-40f4-9f32-f6e881b99e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:50.815643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8e770-867f-4de2-bbd1-c01db4c66751 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:50.816440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ddbc4-aae3-40f4-9f32-f6e881b99e4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:50.816980 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ddbc4-aae3-40f4-9f32-f6e881b99e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:50.817461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ddbc4-aae3-40f4-9f32-f6e881b99e4c admin admin] PATH_INFO: `/v3/users/993d21698b984195bfe32a20fc4f0694` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:50.818947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ddbc4-aae3-40f4-9f32-f6e881b99e4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=993d21698b984195bfe32a20fc4f0694)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:50.821150 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ddbc4-aae3-40f4-9f32-f6e881b99e4c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:50.833355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 999/2016] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:29:50.836427 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8e770-867f-4de2-bbd1-c01db4c66751 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:50.849736 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8e770-867f-4de2-bbd1-c01db4c66751 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:50.857096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520edcb-95b7-41cd-a1d3-68c6995a0589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:50.858114 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520edcb-95b7-41cd-a1d3-68c6995a0589 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:50.858458 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520edcb-95b7-41cd-a1d3-68c6995a0589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:50.858693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520edcb-95b7-41cd-a1d3-68c6995a0589 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:50.859446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520edcb-95b7-41cd-a1d3-68c6995a0589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:50.861471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520edcb-95b7-41cd-a1d3-68c6995a0589 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:50.883293 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ba1ddbc4-aae3-40f4-9f32-f6e881b99e4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '993d21698b984195bfe32a20fc4f0694'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:50.886599 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:50.898217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1000/2017] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:50 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:50.902211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520edcb-95b7-41cd-a1d3-68c6995a0589 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:50.908057 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ba1ddbc4-aae3-40f4-9f32-f6e881b99e4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '993d21698b984195bfe32a20fc4f0694'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:50.910903 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ba1ddbc4-aae3-40f4-9f32-f6e881b99e4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '993d21698b984195bfe32a20fc4f0694'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:50.912555 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ba1ddbc4-aae3-40f4-9f32-f6e881b99e4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '993d21698b984195bfe32a20fc4f0694'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:50.928798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ba1ddbc4-aae3-40f4-9f32-f6e881b99e4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '993d21698b984195bfe32a20fc4f0694'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:50.951872 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1018/2018] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:50 2022] DELETE /identity/v3/users/993d21698b984195bfe32a20fc4f0694 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:29:50.953759 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876a59a-e37f-4c52-aae1-972bba798583 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:50.954701 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876a59a-e37f-4c52-aae1-972bba798583 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:50.954998 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876a59a-e37f-4c52-aae1-972bba798583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:50.990168 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4876a59a-e37f-4c52-aae1-972bba798583 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:50.991408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4876a59a-e37f-4c52-aae1-972bba798583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:51.008720 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520edcb-95b7-41cd-a1d3-68c6995a0589 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:51.052377 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:51.091731 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff43816-9e0f-4fc5-a5c3-acaf83c97089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:51.092552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff43816-9e0f-4fc5-a5c3-acaf83c97089 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:51.092771 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff43816-9e0f-4fc5-a5c3-acaf83c97089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:51.093004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff43816-9e0f-4fc5-a5c3-acaf83c97089 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:51.093706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff43816-9e0f-4fc5-a5c3-acaf83c97089 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:51.094418 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1019/2019] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:51.095647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff43816-9e0f-4fc5-a5c3-acaf83c97089 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:51.100660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876a59a-e37f-4c52-aae1-972bba798583 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:51.102012 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4876a59a-e37f-4c52-aae1-972bba798583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:51.119643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1001/2020] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:50 2022] POST /identity/v3/users => generated 284 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:51.123462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff43816-9e0f-4fc5-a5c3-acaf83c97089 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:51.125721 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-aff43816-9e0f-4fc5-a5c3-acaf83c97089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:51.139877 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff43816-9e0f-4fc5-a5c3-acaf83c97089 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:51.148457 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da81522-3ff6-489a-ab82-a4764d2abd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:51.149188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da81522-3ff6-489a-ab82-a4764d2abd75 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:51.149477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da81522-3ff6-489a-ab82-a4764d2abd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:51.149757 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da81522-3ff6-489a-ab82-a4764d2abd75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:51.154188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da81522-3ff6-489a-ab82-a4764d2abd75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:51.154188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da81522-3ff6-489a-ab82-a4764d2abd75 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:51.166260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1002/2021] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:29:51.170231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da81522-3ff6-489a-ab82-a4764d2abd75 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:51.173854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876a59a-e37f-4c52-aae1-972bba798583 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:51.178279 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4777bd5-c40a-4f26-96a2-d9c667287ab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:51.179056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4777bd5-c40a-4f26-96a2-d9c667287ab5 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:51.179511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4777bd5-c40a-4f26-96a2-d9c667287ab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:51.179884 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4777bd5-c40a-4f26-96a2-d9c667287ab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:51.181382 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da81522-3ff6-489a-ab82-a4764d2abd75 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:51.186687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953873a-7cb6-429a-9d47-be8eb9fd88ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:51.187216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953873a-7cb6-429a-9d47-be8eb9fd88ee admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:51.187505 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953873a-7cb6-429a-9d47-be8eb9fd88ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:51.187786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953873a-7cb6-429a-9d47-be8eb9fd88ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:51.188707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953873a-7cb6-429a-9d47-be8eb9fd88ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:51.190661 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953873a-7cb6-429a-9d47-be8eb9fd88ee admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:51.191478 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4777bd5-c40a-4f26-96a2-d9c667287ab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:51.194006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4777bd5-c40a-4f26-96a2-d9c667287ab5 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:51.204545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1003/2022] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:29:51.207586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953873a-7cb6-429a-9d47-be8eb9fd88ee admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:51.210023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1020/2023] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:29:51.218890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953873a-7cb6-429a-9d47-be8eb9fd88ee admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:51.224452 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b9771-a718-4e4c-b2c0-5f98894f8acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:51.225490 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b9771-a718-4e4c-b2c0-5f98894f8acf admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:51.225834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b9771-a718-4e4c-b2c0-5f98894f8acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:51.226236 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b9771-a718-4e4c-b2c0-5f98894f8acf admin admin] PATH_INFO: `/v3/projects/1752330828e74475a96d782a1aa0b3f6/users/f31b10a9ece749158fff645b7eb0f94a/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:51.243733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b9771-a718-4e4c-b2c0-5f98894f8acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1752330828e74475a96d782a1aa0b3f6, user_id=f31b10a9ece749158fff645b7eb0f94a, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:51.246691 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b9771-a718-4e4c-b2c0-5f98894f8acf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:51.264621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1004/2024] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:51 2022] PUT /identity/v3/projects/1752330828e74475a96d782a1aa0b3f6/users/f31b10a9ece749158fff645b7eb0f94a/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:29:51.274985 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07789ada-6d19-405d-88de-bd4ae1470903 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:51.275286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07789ada-6d19-405d-88de-bd4ae1470903 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:51.275576 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07789ada-6d19-405d-88de-bd4ae1470903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:51.303687 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-07789ada-6d19-405d-88de-bd4ae1470903 None None] MFA Rules not processed for user `f31b10a9ece749158fff645b7eb0f94a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:51.305019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-07789ada-6d19-405d-88de-bd4ae1470903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:51.376491 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:51.392689 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1021/2025] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:51 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:51.396616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b9771-a718-4e4c-b2c0-5f98894f8acf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:51.398601 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c04b9771-a718-4e4c-b2c0-5f98894f8acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:51.497629 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07789ada-6d19-405d-88de-bd4ae1470903 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:51.499753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-07789ada-6d19-405d-88de-bd4ae1470903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:51.502511 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b9771-a718-4e4c-b2c0-5f98894f8acf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:51.507305 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d99db-1f50-4dd8-8ddf-6dd822d9ed92 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cee4d23797e430e9eeddc394ed7f50e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:51.517679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d99db-1f50-4dd8-8ddf-6dd822d9ed92 service glance] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:51.520010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d99db-1f50-4dd8-8ddf-6dd822d9ed92 service glance] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:51.521112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d99db-1f50-4dd8-8ddf-6dd822d9ed92 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:51.541173 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d99db-1f50-4dd8-8ddf-6dd822d9ed92 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:51.548016 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d99db-1f50-4dd8-8ddf-6dd822d9ed92 service glance] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:51.562803 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1005/2026] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:51 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:51.583685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07789ada-6d19-405d-88de-bd4ae1470903 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:51.591276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7845f6-037b-4f71-9927-9c6a3d6c515f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:51.592163 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7845f6-037b-4f71-9927-9c6a3d6c515f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:51.592450 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7845f6-037b-4f71-9927-9c6a3d6c515f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:51.592757 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7845f6-037b-4f71-9927-9c6a3d6c515f admin admin] PATH_INFO: `/v3/projects/5fab27e1dbca4b2194fc9e917868472f` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:51.593964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7845f6-037b-4f71-9927-9c6a3d6c515f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fab27e1dbca4b2194fc9e917868472f)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:51.595912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7845f6-037b-4f71-9927-9c6a3d6c515f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:51.659427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0f7845f6-037b-4f71-9927-9c6a3d6c515f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fab27e1dbca4b2194fc9e917868472f', 'request_id': 'req-0f7845f6-037b-4f71-9927-9c6a3d6c515f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:51.665248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0f7845f6-037b-4f71-9927-9c6a3d6c515f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fab27e1dbca4b2194fc9e917868472f', 'request_id': 'req-0f7845f6-037b-4f71-9927-9c6a3d6c515f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:51.671865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0f7845f6-037b-4f71-9927-9c6a3d6c515f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fab27e1dbca4b2194fc9e917868472f', 'request_id': 'req-0f7845f6-037b-4f71-9927-9c6a3d6c515f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:51.688917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0f7845f6-037b-4f71-9927-9c6a3d6c515f admin admin] The token cache is being invalidate because project 5fab27e1dbca4b2194fc9e917868472f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:51.689221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0f7845f6-037b-4f71-9927-9c6a3d6c515f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:51.689882 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0f7845f6-037b-4f71-9927-9c6a3d6c515f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:51.692795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1022/2027] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:51 2022] DELETE /identity/v3/projects/5fab27e1dbca4b2194fc9e917868472f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:29:51.696479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d99db-1f50-4dd8-8ddf-6dd822d9ed92 service glance] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:51.708677 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d99db-1f50-4dd8-8ddf-6dd822d9ed92 service glance] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:51.766606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:51.775333 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989836c9-873b-4ff8-a1af-37f678367d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:51.776240 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989836c9-873b-4ff8-a1af-37f678367d5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:51.776542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989836c9-873b-4ff8-a1af-37f678367d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:51.776913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989836c9-873b-4ff8-a1af-37f678367d5e admin admin] PATH_INFO: `/v3/users/7f0d36408aac4ee9882aeb99564afe15` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:51.778141 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989836c9-873b-4ff8-a1af-37f678367d5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f0d36408aac4ee9882aeb99564afe15)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:51.780075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989836c9-873b-4ff8-a1af-37f678367d5e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:51.830655 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-989836c9-873b-4ff8-a1af-37f678367d5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f0d36408aac4ee9882aeb99564afe15'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:51.831444 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-989836c9-873b-4ff8-a1af-37f678367d5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f0d36408aac4ee9882aeb99564afe15'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:51.832284 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-989836c9-873b-4ff8-a1af-37f678367d5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f0d36408aac4ee9882aeb99564afe15'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:51.841650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-989836c9-873b-4ff8-a1af-37f678367d5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f0d36408aac4ee9882aeb99564afe15'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:51.855086 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-989836c9-873b-4ff8-a1af-37f678367d5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f0d36408aac4ee9882aeb99564afe15'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:51.872556 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1006/2028] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:51 2022] DELETE /identity/v3/users/7f0d36408aac4ee9882aeb99564afe15 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:52.204937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7845f6-037b-4f71-9927-9c6a3d6c515f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:52.207553 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0f7845f6-037b-4f71-9927-9c6a3d6c515f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:52.220811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7845f6-037b-4f71-9927-9c6a3d6c515f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:52.278454 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf90fb-61b7-469c-b2ec-d3c91d5218e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:52.279252 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf90fb-61b7-469c-b2ec-d3c91d5218e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:52.279541 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf90fb-61b7-469c-b2ec-d3c91d5218e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:52.279823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf90fb-61b7-469c-b2ec-d3c91d5218e7 admin admin] PATH_INFO: `/v3/projects/a361c0842f6c4357815ade2afcd1d377` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:52.281046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf90fb-61b7-469c-b2ec-d3c91d5218e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a361c0842f6c4357815ade2afcd1d377)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:52.283001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf90fb-61b7-469c-b2ec-d3c91d5218e7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:52.318868 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:52.333120 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b8cf90fb-61b7-469c-b2ec-d3c91d5218e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a361c0842f6c4357815ade2afcd1d377', 'request_id': 'req-b8cf90fb-61b7-469c-b2ec-d3c91d5218e7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:52.338576 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b8cf90fb-61b7-469c-b2ec-d3c91d5218e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a361c0842f6c4357815ade2afcd1d377', 'request_id': 'req-b8cf90fb-61b7-469c-b2ec-d3c91d5218e7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:52.344652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b8cf90fb-61b7-469c-b2ec-d3c91d5218e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a361c0842f6c4357815ade2afcd1d377', 'request_id': 'req-b8cf90fb-61b7-469c-b2ec-d3c91d5218e7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:52.357012 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b8cf90fb-61b7-469c-b2ec-d3c91d5218e7 admin admin] The token cache is being invalidate because project a361c0842f6c4357815ade2afcd1d377 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:52.357305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b8cf90fb-61b7-469c-b2ec-d3c91d5218e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:52.357933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b8cf90fb-61b7-469c-b2ec-d3c91d5218e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:52.359481 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1023/2029] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:52 2022] DELETE /identity/v3/projects/a361c0842f6c4357815ade2afcd1d377 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:52.365007 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8f80f-6599-4890-8e8f-f486f24fb03a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:52.365382 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8f80f-6599-4890-8e8f-f486f24fb03a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:52.365678 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8f80f-6599-4890-8e8f-f486f24fb03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:52.383970 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e0e8f80f-6599-4890-8e8f-f486f24fb03a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:52.385210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e0e8f80f-6599-4890-8e8f-f486f24fb03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:52.442694 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1007/2030] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:29:52.450382 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32664fe1-c338-44b9-ab21-bfe16923c8b7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:52.450700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32664fe1-c338-44b9-ab21-bfe16923c8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:52.450980 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32664fe1-c338-44b9-ab21-bfe16923c8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:52.469249 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-32664fe1-c338-44b9-ab21-bfe16923c8b7 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:52.470456 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-32664fe1-c338-44b9-ab21-bfe16923c8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:52.478504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1024/2031] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:52.483003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8f80f-6599-4890-8e8f-f486f24fb03a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:52.493343 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8f80f-6599-4890-8e8f-f486f24fb03a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:52.498068 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609ff32-5e7a-4fd8-b142-a2724c97e269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:52.498795 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609ff32-5e7a-4fd8-b142-a2724c97e269 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:52.499078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609ff32-5e7a-4fd8-b142-a2724c97e269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:52.499360 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609ff32-5e7a-4fd8-b142-a2724c97e269 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:52.500095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609ff32-5e7a-4fd8-b142-a2724c97e269 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:52.500414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609ff32-5e7a-4fd8-b142-a2724c97e269 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:52.502053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609ff32-5e7a-4fd8-b142-a2724c97e269 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:52.514509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1008/2032] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:52.518840 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32664fe1-c338-44b9-ab21-bfe16923c8b7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:52.528257 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32664fe1-c338-44b9-ab21-bfe16923c8b7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:52.533048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846710a-0c35-44d0-8a28-95f5881c8275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:52.533630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846710a-0c35-44d0-8a28-95f5881c8275 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:52.533912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846710a-0c35-44d0-8a28-95f5881c8275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:52.534184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846710a-0c35-44d0-8a28-95f5881c8275 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:52.534911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846710a-0c35-44d0-8a28-95f5881c8275 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:52.536836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8846710a-0c35-44d0-8a28-95f5881c8275 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:52.561909 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1025/2033] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:52 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:29:52.566647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609ff32-5e7a-4fd8-b142-a2724c97e269 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:52.577109 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609ff32-5e7a-4fd8-b142-a2724c97e269 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:52.638490 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e935cf1-4247-458c-a761-85078d719d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:52.639164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e935cf1-4247-458c-a761-85078d719d0f admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:52.639452 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e935cf1-4247-458c-a761-85078d719d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:52.639730 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e935cf1-4247-458c-a761-85078d719d0f admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:52.640429 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e935cf1-4247-458c-a761-85078d719d0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:52.642504 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e935cf1-4247-458c-a761-85078d719d0f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:52.650741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:52.665149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1009/2034] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:52 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:52.669185 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846710a-0c35-44d0-8a28-95f5881c8275 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:52.679229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8846710a-0c35-44d0-8a28-95f5881c8275 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:52.683748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f675ed7-d25a-4b36-aaeb-2c840eb23500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:52.684401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f675ed7-d25a-4b36-aaeb-2c840eb23500 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:52.684709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f675ed7-d25a-4b36-aaeb-2c840eb23500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:52.685004 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f675ed7-d25a-4b36-aaeb-2c840eb23500 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:52.685709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f675ed7-d25a-4b36-aaeb-2c840eb23500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:52.687299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f675ed7-d25a-4b36-aaeb-2c840eb23500 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:52.697418 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1026/2035] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:52.701087 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e935cf1-4247-458c-a761-85078d719d0f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:52.711688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e935cf1-4247-458c-a761-85078d719d0f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:52.716387 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a799df-5e8e-442c-b380-fbcb5798e8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:52.717081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a799df-5e8e-442c-b380-fbcb5798e8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:52.717364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a799df-5e8e-442c-b380-fbcb5798e8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:52.717643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a799df-5e8e-442c-b380-fbcb5798e8e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:52.718370 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a799df-5e8e-442c-b380-fbcb5798e8e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:52.719957 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a799df-5e8e-442c-b380-fbcb5798e8e5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:52.730586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1010/2036] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:52.734482 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f675ed7-d25a-4b36-aaeb-2c840eb23500 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:52.744583 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f675ed7-d25a-4b36-aaeb-2c840eb23500 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:52.749170 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2c4d5-ee2a-4efa-b7a0-f619554e6d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:52.750038 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2c4d5-ee2a-4efa-b7a0-f619554e6d0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:52.750322 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2c4d5-ee2a-4efa-b7a0-f619554e6d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:52.750598 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2c4d5-ee2a-4efa-b7a0-f619554e6d0b admin admin] PATH_INFO: `/v3/projects/1674901c25c74b63a834b11652e31149/users/0ba504cd4cd649dda79ba3a2112c45ef/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:52.765554 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2c4d5-ee2a-4efa-b7a0-f619554e6d0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1674901c25c74b63a834b11652e31149, user_id=0ba504cd4cd649dda79ba3a2112c45ef, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:52.768329 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2c4d5-ee2a-4efa-b7a0-f619554e6d0b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:52.782882 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1027/2037] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:52 2022] PUT /identity/v3/projects/1674901c25c74b63a834b11652e31149/users/0ba504cd4cd649dda79ba3a2112c45ef/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:52.787985 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac74dc-de13-4757-a4f0-bb2e3ef46a13 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:52.788238 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac74dc-de13-4757-a4f0-bb2e3ef46a13 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:52.788546 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac74dc-de13-4757-a4f0-bb2e3ef46a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:52.806011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-35ac74dc-de13-4757-a4f0-bb2e3ef46a13 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:52.807209 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-35ac74dc-de13-4757-a4f0-bb2e3ef46a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:52.865120 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1011/2038] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:52.874656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2c4d5-ee2a-4efa-b7a0-f619554e6d0b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:52.876340 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6ee2c4d5-ee2a-4efa-b7a0-f619554e6d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:52.932674 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2c4d5-ee2a-4efa-b7a0-f619554e6d0b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:52.936876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d0354-3683-4194-912c-0717471079c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:52.937541 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d0354-3683-4194-912c-0717471079c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:52.937835 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d0354-3683-4194-912c-0717471079c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:52.938194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d0354-3683-4194-912c-0717471079c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:52.950833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d0354-3683-4194-912c-0717471079c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:52.952406 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d0354-3683-4194-912c-0717471079c3 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:52.964741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1028/2039] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:53.677612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2273847-f10f-4494-8850-42f56cf44891 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:53.677910 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2273847-f10f-4494-8850-42f56cf44891 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:53.678199 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2273847-f10f-4494-8850-42f56cf44891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:53.695990 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f2273847-f10f-4494-8850-42f56cf44891 None None] MFA Rules not processed for user `bc233897770f47e5bd40fb03afabe49c`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:53.697242 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f2273847-f10f-4494-8850-42f56cf44891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:53.757406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1012/2040] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:53.994101 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebcc5c-2c90-41d9-9730-f53a68ba875e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:53.995239 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebcc5c-2c90-41d9-9730-f53a68ba875e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:53.995606 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebcc5c-2c90-41d9-9730-f53a68ba875e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:54.045784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bbebcc5c-2c90-41d9-9730-f53a68ba875e None None] MFA Rules not processed for user `0ba504cd4cd649dda79ba3a2112c45ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:54.047330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bbebcc5c-2c90-41d9-9730-f53a68ba875e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:54.125157 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:54.196721 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1029/2041] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:54.202532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2273847-f10f-4494-8850-42f56cf44891 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:54.224237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2273847-f10f-4494-8850-42f56cf44891 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:54.243938 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4905df-6a3e-4ab2-a498-3b3ede048302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:54.244539 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4905df-6a3e-4ab2-a498-3b3ede048302 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:54.244906 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4905df-6a3e-4ab2-a498-3b3ede048302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:54.245242 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4905df-6a3e-4ab2-a498-3b3ede048302 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:54.248127 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4905df-6a3e-4ab2-a498-3b3ede048302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:54.250252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4905df-6a3e-4ab2-a498-3b3ede048302 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:54.320057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1013/2042] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:54 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:54.324238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebcc5c-2c90-41d9-9730-f53a68ba875e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:54.344858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebcc5c-2c90-41d9-9730-f53a68ba875e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:54.413334 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5282014-b4f5-4863-94b9-ebdf16219944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:54.414021 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5282014-b4f5-4863-94b9-ebdf16219944 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:54.414329 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5282014-b4f5-4863-94b9-ebdf16219944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:54.414592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5282014-b4f5-4863-94b9-ebdf16219944 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:54.415315 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5282014-b4f5-4863-94b9-ebdf16219944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:54.417547 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5282014-b4f5-4863-94b9-ebdf16219944 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:54.444656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1030/2043] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:54 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:54.447221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4905df-6a3e-4ab2-a498-3b3ede048302 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:54.469434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4905df-6a3e-4ab2-a498-3b3ede048302 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:54.475397 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334c7b7-9fec-45a7-9bcb-a06c9557010a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:54.475808 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334c7b7-9fec-45a7-9bcb-a06c9557010a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:54.476090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334c7b7-9fec-45a7-9bcb-a06c9557010a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:54.476365 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334c7b7-9fec-45a7-9bcb-a06c9557010a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:54.477270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334c7b7-9fec-45a7-9bcb-a06c9557010a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:54.479362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334c7b7-9fec-45a7-9bcb-a06c9557010a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:54.491599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1014/2044] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:29:54.495793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5282014-b4f5-4863-94b9-ebdf16219944 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:54.507981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5282014-b4f5-4863-94b9-ebdf16219944 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:54.513087 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d6a2c-ba6d-477b-8fa4-36d0729ba558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:54.513998 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d6a2c-ba6d-477b-8fa4-36d0729ba558 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:54.514291 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d6a2c-ba6d-477b-8fa4-36d0729ba558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:54.514573 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d6a2c-ba6d-477b-8fa4-36d0729ba558 admin admin] PATH_INFO: `/v3/projects/d33d7e2911114bd8b4364e6703069459/users/b2b483418bb946528522158286499bd4/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:54.537269 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d6a2c-ba6d-477b-8fa4-36d0729ba558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d33d7e2911114bd8b4364e6703069459, user_id=b2b483418bb946528522158286499bd4, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:54.539980 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d6a2c-ba6d-477b-8fa4-36d0729ba558 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:54.556550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1031/2045] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:54 2022] PUT /identity/v3/projects/d33d7e2911114bd8b4364e6703069459/users/b2b483418bb946528522158286499bd4/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:54.560327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334c7b7-9fec-45a7-9bcb-a06c9557010a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:54.574086 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334c7b7-9fec-45a7-9bcb-a06c9557010a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:54.648367 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:54.661614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b966f1f2-af4f-42a2-90c8-181d12e67c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:54.662787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966f1f2-af4f-42a2-90c8-181d12e67c5f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:54.663202 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966f1f2-af4f-42a2-90c8-181d12e67c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:54.663488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b966f1f2-af4f-42a2-90c8-181d12e67c5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:54.664363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b966f1f2-af4f-42a2-90c8-181d12e67c5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:54.666091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b966f1f2-af4f-42a2-90c8-181d12e67c5f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:54.684617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1015/2046] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:54.685831 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d6a2c-ba6d-477b-8fa4-36d0729ba558 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:54.699368 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d6a2c-ba6d-477b-8fa4-36d0729ba558 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:54.706306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfe3f9-9d2c-4aa6-96a5-7cb3761b9f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:54.707477 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfe3f9-9d2c-4aa6-96a5-7cb3761b9f38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:54.707966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfe3f9-9d2c-4aa6-96a5-7cb3761b9f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:54.708445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfe3f9-9d2c-4aa6-96a5-7cb3761b9f38 admin admin] PATH_INFO: `/v3/projects/d33d7e2911114bd8b4364e6703069459/users/b2b483418bb946528522158286499bd4/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:54.713161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cfe3f9-9d2c-4aa6-96a5-7cb3761b9f38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d33d7e2911114bd8b4364e6703069459, user_id=b2b483418bb946528522158286499bd4, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:54.715914 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cfe3f9-9d2c-4aa6-96a5-7cb3761b9f38 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:54.741089 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1032/2047] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:54 2022] PUT /identity/v3/projects/d33d7e2911114bd8b4364e6703069459/users/b2b483418bb946528522158286499bd4/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:55.464747 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ca081-645f-44bd-bca1-1a32e5cf1587 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:55.465342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ca081-645f-44bd-bca1-1a32e5cf1587 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:55.465342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ca081-645f-44bd-bca1-1a32e5cf1587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:55.497501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7e6ca081-645f-44bd-bca1-1a32e5cf1587 None None] MFA Rules not processed for user `b2b483418bb946528522158286499bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:55.498630 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7e6ca081-645f-44bd-bca1-1a32e5cf1587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:55.580210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1016/2048] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:55.592450 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfe3f9-9d2c-4aa6-96a5-7cb3761b9f38 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:55.594260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-22cfe3f9-9d2c-4aa6-96a5-7cb3761b9f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:55.665507 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfe3f9-9d2c-4aa6-96a5-7cb3761b9f38 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:55.670123 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33815050-7b27-4c70-bf46-ff388536f7c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:55.670685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33815050-7b27-4c70-bf46-ff388536f7c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:55.670971 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33815050-7b27-4c70-bf46-ff388536f7c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:55.671246 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33815050-7b27-4c70-bf46-ff388536f7c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:55.781023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:55.789855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33815050-7b27-4c70-bf46-ff388536f7c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:55.791327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33815050-7b27-4c70-bf46-ff388536f7c4 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:55.817453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1033/2049] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:56.988222 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ca081-645f-44bd-bca1-1a32e5cf1587 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:56.989288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7e6ca081-645f-44bd-bca1-1a32e5cf1587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:57.058878 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ca081-645f-44bd-bca1-1a32e5cf1587 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:57.064381 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db131f4a-39fb-4179-b4a4-aeb2a0d88559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:57.065289 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db131f4a-39fb-4179-b4a4-aeb2a0d88559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:57.065578 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db131f4a-39fb-4179-b4a4-aeb2a0d88559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:57.065855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db131f4a-39fb-4179-b4a4-aeb2a0d88559 admin admin] PATH_INFO: `/v3/users/653aa49b3a3e4171a1b285bb4c153c2e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:57.067054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db131f4a-39fb-4179-b4a4-aeb2a0d88559 admin admin] RBAC: Authorizing `identity:delete_user(user_id=653aa49b3a3e4171a1b285bb4c153c2e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:57.069018 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db131f4a-39fb-4179-b4a4-aeb2a0d88559 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:57.117750 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db131f4a-39fb-4179-b4a4-aeb2a0d88559 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '653aa49b3a3e4171a1b285bb4c153c2e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:57.118412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db131f4a-39fb-4179-b4a4-aeb2a0d88559 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '653aa49b3a3e4171a1b285bb4c153c2e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:57.119023 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db131f4a-39fb-4179-b4a4-aeb2a0d88559 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '653aa49b3a3e4171a1b285bb4c153c2e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:57.125973 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db131f4a-39fb-4179-b4a4-aeb2a0d88559 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '653aa49b3a3e4171a1b285bb4c153c2e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:57.140651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db131f4a-39fb-4179-b4a4-aeb2a0d88559 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '653aa49b3a3e4171a1b285bb4c153c2e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:57.143404 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:57.159789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1017/2050] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:56 2022] DELETE /identity/v3/users/653aa49b3a3e4171a1b285bb4c153c2e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:29:57.164375 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6abbdd-2e9d-45c9-809c-38d8f3b832a6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:57.164830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6abbdd-2e9d-45c9-809c-38d8f3b832a6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:57.164963 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6abbdd-2e9d-45c9-809c-38d8f3b832a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:57.188742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9d6abbdd-2e9d-45c9-809c-38d8f3b832a6 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:57.189959 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9d6abbdd-2e9d-45c9-809c-38d8f3b832a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:57.234782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db131f4a-39fb-4179-b4a4-aeb2a0d88559 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:57.236185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-db131f4a-39fb-4179-b4a4-aeb2a0d88559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:57.274894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1034/2051] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:57.286283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6abbdd-2e9d-45c9-809c-38d8f3b832a6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:57.288652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9d6abbdd-2e9d-45c9-809c-38d8f3b832a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:57.308704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db131f4a-39fb-4179-b4a4-aeb2a0d88559 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:57.312485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7112d-9737-42c0-9d4b-c3b466630c61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:57.313162 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7112d-9737-42c0-9d4b-c3b466630c61 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:57.313447 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7112d-9737-42c0-9d4b-c3b466630c61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:57.313781 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7112d-9737-42c0-9d4b-c3b466630c61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:57.316279 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cce7112d-9737-42c0-9d4b-c3b466630c61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:57.351395 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6abbdd-2e9d-45c9-809c-38d8f3b832a6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:57.355974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceaa02f-2c56-4a3a-ab1d-3b46f0cdfa9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:57.358264 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaa02f-2c56-4a3a-ab1d-3b46f0cdfa9d service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:57.358549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaa02f-2c56-4a3a-ab1d-3b46f0cdfa9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:57.358830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaa02f-2c56-4a3a-ab1d-3b46f0cdfa9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:57.368694 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceaa02f-2c56-4a3a-ab1d-3b46f0cdfa9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:57.370137 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceaa02f-2c56-4a3a-ab1d-3b46f0cdfa9d service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:57.380849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1035/2052] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:29:57.388250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7112d-9737-42c0-9d4b-c3b466630c61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:57.392658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7112d-9737-42c0-9d4b-c3b466630c61 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:57.403013 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1018/2053] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:29:57.682097 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceaa02f-2c56-4a3a-ab1d-3b46f0cdfa9d service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:57.683606 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0ceaa02f-2c56-4a3a-ab1d-3b46f0cdfa9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:57.697758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceaa02f-2c56-4a3a-ab1d-3b46f0cdfa9d service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:57.702455 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529d49c-cd2d-4b5e-a342-6e9d19cd2016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:57.703345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529d49c-cd2d-4b5e-a342-6e9d19cd2016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:57.703633 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529d49c-cd2d-4b5e-a342-6e9d19cd2016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:57.703914 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529d49c-cd2d-4b5e-a342-6e9d19cd2016 admin admin] PATH_INFO: `/v3/projects/222e4821207649d0b9b40e7f96b34873` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:57.705121 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529d49c-cd2d-4b5e-a342-6e9d19cd2016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=222e4821207649d0b9b40e7f96b34873)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:57.707549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529d49c-cd2d-4b5e-a342-6e9d19cd2016 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:57.754506 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f529d49c-cd2d-4b5e-a342-6e9d19cd2016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '222e4821207649d0b9b40e7f96b34873', 'request_id': 'req-f529d49c-cd2d-4b5e-a342-6e9d19cd2016'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:57.760134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f529d49c-cd2d-4b5e-a342-6e9d19cd2016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '222e4821207649d0b9b40e7f96b34873', 'request_id': 'req-f529d49c-cd2d-4b5e-a342-6e9d19cd2016'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:57.766703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f529d49c-cd2d-4b5e-a342-6e9d19cd2016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '222e4821207649d0b9b40e7f96b34873', 'request_id': 'req-f529d49c-cd2d-4b5e-a342-6e9d19cd2016'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:57.780887 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f529d49c-cd2d-4b5e-a342-6e9d19cd2016 admin admin] The token cache is being invalidate because project 222e4821207649d0b9b40e7f96b34873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:57.781181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f529d49c-cd2d-4b5e-a342-6e9d19cd2016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:57.781845 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f529d49c-cd2d-4b5e-a342-6e9d19cd2016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:57.783502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1036/2054] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:57 2022] DELETE /identity/v3/projects/222e4821207649d0b9b40e7f96b34873 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:57.788401 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7112d-9737-42c0-9d4b-c3b466630c61 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:57.799473 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7112d-9737-42c0-9d4b-c3b466630c61 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:57.863882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76309e-9ecc-4115-8dbe-c2160fb9ef6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:57.864308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76309e-9ecc-4115-8dbe-c2160fb9ef6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:57.864680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76309e-9ecc-4115-8dbe-c2160fb9ef6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:57.865013 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76309e-9ecc-4115-8dbe-c2160fb9ef6a admin admin] PATH_INFO: `/v3/users/842cdc98597c41a7bec88a68e23f64e6` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:57.866280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76309e-9ecc-4115-8dbe-c2160fb9ef6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=842cdc98597c41a7bec88a68e23f64e6)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:57.868384 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76309e-9ecc-4115-8dbe-c2160fb9ef6a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:57.923032 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=8,keystone:SELECT=92 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:57.926149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8c76309e-9ecc-4115-8dbe-c2160fb9ef6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '842cdc98597c41a7bec88a68e23f64e6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:57.927318 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8c76309e-9ecc-4115-8dbe-c2160fb9ef6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '842cdc98597c41a7bec88a68e23f64e6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:57.928528 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8c76309e-9ecc-4115-8dbe-c2160fb9ef6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '842cdc98597c41a7bec88a68e23f64e6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:57.936864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8c76309e-9ecc-4115-8dbe-c2160fb9ef6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '842cdc98597c41a7bec88a68e23f64e6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:57.949152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8c76309e-9ecc-4115-8dbe-c2160fb9ef6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '842cdc98597c41a7bec88a68e23f64e6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:57.966545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1019/2055] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:57 2022] DELETE /identity/v3/users/842cdc98597c41a7bec88a68e23f64e6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:58.259686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529d49c-cd2d-4b5e-a342-6e9d19cd2016 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:58.261538 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f529d49c-cd2d-4b5e-a342-6e9d19cd2016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:58.282660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529d49c-cd2d-4b5e-a342-6e9d19cd2016 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:58.308175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:58.350739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2440a-8d97-4a8e-99de-e8dd93ceafa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:58.351942 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2440a-8d97-4a8e-99de-e8dd93ceafa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:58.352232 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2440a-8d97-4a8e-99de-e8dd93ceafa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:58.352563 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2440a-8d97-4a8e-99de-e8dd93ceafa8 admin admin] PATH_INFO: `/v3/projects/770bdaf40ae74972ba29f8b847fa05ea` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:58.356571 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a2440a-8d97-4a8e-99de-e8dd93ceafa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=770bdaf40ae74972ba29f8b847fa05ea)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:58.358613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a2440a-8d97-4a8e-99de-e8dd93ceafa8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:58.416226 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-93a2440a-8d97-4a8e-99de-e8dd93ceafa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '770bdaf40ae74972ba29f8b847fa05ea', 'request_id': 'req-93a2440a-8d97-4a8e-99de-e8dd93ceafa8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:58.425822 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-93a2440a-8d97-4a8e-99de-e8dd93ceafa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '770bdaf40ae74972ba29f8b847fa05ea', 'request_id': 'req-93a2440a-8d97-4a8e-99de-e8dd93ceafa8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:58.441020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-93a2440a-8d97-4a8e-99de-e8dd93ceafa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '770bdaf40ae74972ba29f8b847fa05ea', 'request_id': 'req-93a2440a-8d97-4a8e-99de-e8dd93ceafa8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:58.460446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-93a2440a-8d97-4a8e-99de-e8dd93ceafa8 admin admin] The token cache is being invalidate because project 770bdaf40ae74972ba29f8b847fa05ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:29:58.461995 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-93a2440a-8d97-4a8e-99de-e8dd93ceafa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:58.462690 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-93a2440a-8d97-4a8e-99de-e8dd93ceafa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:58.466760 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1037/2056] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:29:58 2022] DELETE /identity/v3/projects/770bdaf40ae74972ba29f8b847fa05ea => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:58.486890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989949f-3f88-4c80-808a-34386b95a02e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:58.486890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989949f-3f88-4c80-808a-34386b95a02e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:58.486890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989949f-3f88-4c80-808a-34386b95a02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:58.502218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9989949f-3f88-4c80-808a-34386b95a02e None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:58.504065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9989949f-3f88-4c80-808a-34386b95a02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:58.575569 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1020/2057] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:58.583569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886ec47-cec9-42a4-a2be-68de10c41637 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:58.583979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886ec47-cec9-42a4-a2be-68de10c41637 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:58.584338 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886ec47-cec9-42a4-a2be-68de10c41637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:58.608790 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6886ec47-cec9-42a4-a2be-68de10c41637 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:58.617630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6886ec47-cec9-42a4-a2be-68de10c41637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:58.631253 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1038/2058] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:58.635451 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989949f-3f88-4c80-808a-34386b95a02e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:58.652924 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989949f-3f88-4c80-808a-34386b95a02e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:58.659141 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73e1d1-0ca6-44ee-8c5a-6f07e335626b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:58.659901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73e1d1-0ca6-44ee-8c5a-6f07e335626b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:58.660254 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73e1d1-0ca6-44ee-8c5a-6f07e335626b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:58.660645 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73e1d1-0ca6-44ee-8c5a-6f07e335626b admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:58.661637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73e1d1-0ca6-44ee-8c5a-6f07e335626b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:29:58.662726 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73e1d1-0ca6-44ee-8c5a-6f07e335626b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:58.664348 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73e1d1-0ca6-44ee-8c5a-6f07e335626b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:58.684166 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1021/2059] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:29:58.688497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886ec47-cec9-42a4-a2be-68de10c41637 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:58.699447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886ec47-cec9-42a4-a2be-68de10c41637 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:58.704549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1df65-5781-4d51-8ec3-b5a4226291b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:58.705206 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1df65-5781-4d51-8ec3-b5a4226291b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:58.705497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1df65-5781-4d51-8ec3-b5a4226291b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:58.705782 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1df65-5781-4d51-8ec3-b5a4226291b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:58.706542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea1df65-5781-4d51-8ec3-b5a4226291b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:58.708767 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea1df65-5781-4d51-8ec3-b5a4226291b9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:58.745568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1039/2060] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:29:58 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:29:58.747153 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73e1d1-0ca6-44ee-8c5a-6f07e335626b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:58.760045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73e1d1-0ca6-44ee-8c5a-6f07e335626b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:58.840540 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198cf9e-2259-4ce8-b812-f9b69d355324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:58.841440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198cf9e-2259-4ce8-b812-f9b69d355324 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:58.842120 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198cf9e-2259-4ce8-b812-f9b69d355324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:58.842120 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198cf9e-2259-4ce8-b812-f9b69d355324 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:58.843712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198cf9e-2259-4ce8-b812-f9b69d355324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:58.845707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198cf9e-2259-4ce8-b812-f9b69d355324 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:58.881128 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1022/2061] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:29:58 2022] POST /identity/v3/users => generated 292 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:29:58.885824 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1df65-5781-4d51-8ec3-b5a4226291b9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:58.898402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1df65-5781-4d51-8ec3-b5a4226291b9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:58.903909 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48238120-97fb-49c7-98d9-b411227b5f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:58.904762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48238120-97fb-49c7-98d9-b411227b5f9c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:58.905148 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48238120-97fb-49c7-98d9-b411227b5f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:58.905436 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48238120-97fb-49c7-98d9-b411227b5f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:58.906202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48238120-97fb-49c7-98d9-b411227b5f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:58.908019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48238120-97fb-49c7-98d9-b411227b5f9c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:58.946419 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1040/2062] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:58.950898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198cf9e-2259-4ce8-b812-f9b69d355324 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:58.972448 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198cf9e-2259-4ce8-b812-f9b69d355324 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:58.981837 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f775e-3c6c-46e0-b5c8-de1d91df12e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:58.982520 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f775e-3c6c-46e0-b5c8-de1d91df12e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:58.982805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f775e-3c6c-46e0-b5c8-de1d91df12e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:58.983081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f775e-3c6c-46e0-b5c8-de1d91df12e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:58.983815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f775e-3c6c-46e0-b5c8-de1d91df12e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:58.985630 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f775e-3c6c-46e0-b5c8-de1d91df12e9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:59.004762 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1023/2063] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:29:59.006384 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48238120-97fb-49c7-98d9-b411227b5f9c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:59.023833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48238120-97fb-49c7-98d9-b411227b5f9c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:59.036374 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc58e74-8d54-4214-8674-b09f3be28fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:59.037323 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc58e74-8d54-4214-8674-b09f3be28fb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:59.037616 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc58e74-8d54-4214-8674-b09f3be28fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:59.039444 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc58e74-8d54-4214-8674-b09f3be28fb3 admin admin] PATH_INFO: `/v3/projects/9b907776422d4422af7a1c119ca53ffe/users/7c86ad1e125b4134b8e59152765aad67/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:59.075799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc58e74-8d54-4214-8674-b09f3be28fb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b907776422d4422af7a1c119ca53ffe, user_id=7c86ad1e125b4134b8e59152765aad67, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:59.078791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc58e74-8d54-4214-8674-b09f3be28fb3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:59.109253 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1041/2064] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:29:59 2022] PUT /identity/v3/projects/9b907776422d4422af7a1c119ca53ffe/users/7c86ad1e125b4134b8e59152765aad67/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:29:59.123408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834d81e-ba5a-417d-9be9-743243c70135 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:59.123767 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834d81e-ba5a-417d-9be9-743243c70135 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:59.124060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834d81e-ba5a-417d-9be9-743243c70135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:59.171329 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4834d81e-ba5a-417d-9be9-743243c70135 None None] MFA Rules not processed for user `7c86ad1e125b4134b8e59152765aad67`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:59.172568 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4834d81e-ba5a-417d-9be9-743243c70135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:59.200598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:59.257727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1024/2065] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:29:59.269524 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc58e74-8d54-4214-8674-b09f3be28fb3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:59.271548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0dc58e74-8d54-4214-8674-b09f3be28fb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:59.349344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:59.355543 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc58e74-8d54-4214-8674-b09f3be28fb3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:59.361269 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea3d70-e7e4-456c-8584-7ecca4f5a18d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:59.362637 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea3d70-e7e4-456c-8584-7ecca4f5a18d service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:59.363226 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea3d70-e7e4-456c-8584-7ecca4f5a18d service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:59.363838 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea3d70-e7e4-456c-8584-7ecca4f5a18d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:59.373362 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea3d70-e7e4-456c-8584-7ecca4f5a18d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:59.374738 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea3d70-e7e4-456c-8584-7ecca4f5a18d service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:59.385363 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1042/2066] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:29:59.405687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834d81e-ba5a-417d-9be9-743243c70135 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:59.488149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834d81e-ba5a-417d-9be9-743243c70135 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:59.496288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47f2bf-50d2-4e9c-8647-080223de6375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:59.497832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47f2bf-50d2-4e9c-8647-080223de6375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:59.498317 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47f2bf-50d2-4e9c-8647-080223de6375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:59.498614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47f2bf-50d2-4e9c-8647-080223de6375 admin admin] PATH_INFO: `/v3/users/7c86ad1e125b4134b8e59152765aad67` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:59.500656 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47f2bf-50d2-4e9c-8647-080223de6375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c86ad1e125b4134b8e59152765aad67)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:59.504750 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47f2bf-50d2-4e9c-8647-080223de6375 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:59.581410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-af47f2bf-50d2-4e9c-8647-080223de6375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c86ad1e125b4134b8e59152765aad67'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:59.582127 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-af47f2bf-50d2-4e9c-8647-080223de6375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c86ad1e125b4134b8e59152765aad67'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:59.582793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-af47f2bf-50d2-4e9c-8647-080223de6375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c86ad1e125b4134b8e59152765aad67'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:59.590702 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-af47f2bf-50d2-4e9c-8647-080223de6375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c86ad1e125b4134b8e59152765aad67'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:59.610285 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-af47f2bf-50d2-4e9c-8647-080223de6375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c86ad1e125b4134b8e59152765aad67'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:29:59.624737 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1025/2067] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:29:59 2022] DELETE /identity/v3/users/7c86ad1e125b4134b8e59152765aad67 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:29:59.629221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d313e-573f-4c56-b0a2-38b7a471ade3 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:59.629641 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d313e-573f-4c56-b0a2-38b7a471ade3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:59.630015 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d313e-573f-4c56-b0a2-38b7a471ade3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:59.651441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5b5d313e-573f-4c56-b0a2-38b7a471ade3 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:29:59.653067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5b5d313e-573f-4c56-b0a2-38b7a471ade3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:59.720169 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1043/2068] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:29:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:29:59.730552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47f2bf-50d2-4e9c-8647-080223de6375 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:29:59.731978 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-af47f2bf-50d2-4e9c-8647-080223de6375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:29:59.779910 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:29:59.798263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47f2bf-50d2-4e9c-8647-080223de6375 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:29:59.802739 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c6166-ad1f-436c-bcc9-9f783088140c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:29:59.803405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c6166-ad1f-436c-bcc9-9f783088140c service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:29:59.803696 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c6166-ad1f-436c-bcc9-9f783088140c service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:29:59.804074 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c6166-ad1f-436c-bcc9-9f783088140c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:29:59.814454 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c6166-ad1f-436c-bcc9-9f783088140c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:29:59.815964 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c6166-ad1f-436c-bcc9-9f783088140c service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:29:59.826028 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1026/2069] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:29:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:00.176200 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d313e-573f-4c56-b0a2-38b7a471ade3 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:00.179893 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5b5d313e-573f-4c56-b0a2-38b7a471ade3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:00.199172 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d313e-573f-4c56-b0a2-38b7a471ade3 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:00.216127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438fde91-160e-4e69-9762-7ad7ca262fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:00.217072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fde91-160e-4e69-9762-7ad7ca262fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:00.217365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fde91-160e-4e69-9762-7ad7ca262fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:00.217647 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438fde91-160e-4e69-9762-7ad7ca262fd3 admin admin] PATH_INFO: `/v3/projects/9b907776422d4422af7a1c119ca53ffe` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:00.227016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438fde91-160e-4e69-9762-7ad7ca262fd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b907776422d4422af7a1c119ca53ffe)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:00.234444 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438fde91-160e-4e69-9762-7ad7ca262fd3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:00.320896 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-438fde91-160e-4e69-9762-7ad7ca262fd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b907776422d4422af7a1c119ca53ffe', 'request_id': 'req-438fde91-160e-4e69-9762-7ad7ca262fd3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:00.333684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-438fde91-160e-4e69-9762-7ad7ca262fd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b907776422d4422af7a1c119ca53ffe', 'request_id': 'req-438fde91-160e-4e69-9762-7ad7ca262fd3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:00.347502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-438fde91-160e-4e69-9762-7ad7ca262fd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b907776422d4422af7a1c119ca53ffe', 'request_id': 'req-438fde91-160e-4e69-9762-7ad7ca262fd3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:00.381786 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-438fde91-160e-4e69-9762-7ad7ca262fd3 admin admin] The token cache is being invalidate because project 9b907776422d4422af7a1c119ca53ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:00.382985 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-438fde91-160e-4e69-9762-7ad7ca262fd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:00.383912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-438fde91-160e-4e69-9762-7ad7ca262fd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:00.389698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1044/2070] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:00 2022] DELETE /identity/v3/projects/9b907776422d4422af7a1c119ca53ffe => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:00.394558 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54eed5-d6d9-416d-a5b9-61a793e0b7ff None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:00.395051 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54eed5-d6d9-416d-a5b9-61a793e0b7ff None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:00.395390 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54eed5-d6d9-416d-a5b9-61a793e0b7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:00.418542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8f54eed5-d6d9-416d-a5b9-61a793e0b7ff None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:00.419823 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8f54eed5-d6d9-416d-a5b9-61a793e0b7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:00.489789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1027/2071] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:00.497632 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df0d5a-fd23-487f-84e2-9c8931f305e5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:00.497928 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df0d5a-fd23-487f-84e2-9c8931f305e5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:00.498211 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df0d5a-fd23-487f-84e2-9c8931f305e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:00.519586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-66df0d5a-fd23-487f-84e2-9c8931f305e5 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:00.521105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-66df0d5a-fd23-487f-84e2-9c8931f305e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:00.531048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1045/2072] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:00.535711 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54eed5-d6d9-416d-a5b9-61a793e0b7ff None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:00.552643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54eed5-d6d9-416d-a5b9-61a793e0b7ff None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:00.564322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fca0da-013e-478c-bcf2-e9713a12e0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:00.565913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fca0da-013e-478c-bcf2-e9713a12e0c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:00.566521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fca0da-013e-478c-bcf2-e9713a12e0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:00.566911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fca0da-013e-478c-bcf2-e9713a12e0c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:00.567908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fca0da-013e-478c-bcf2-e9713a12e0c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:30:00.568371 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fca0da-013e-478c-bcf2-e9713a12e0c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:00.571871 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fca0da-013e-478c-bcf2-e9713a12e0c5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:00.597932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1028/2073] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:30:00.598716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df0d5a-fd23-487f-84e2-9c8931f305e5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:00.614925 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df0d5a-fd23-487f-84e2-9c8931f305e5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:00.632114 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833ae84-b73d-4753-bf34-7c819cfea483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:00.633066 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833ae84-b73d-4753-bf34-7c819cfea483 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:00.635436 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833ae84-b73d-4753-bf34-7c819cfea483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:00.635875 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833ae84-b73d-4753-bf34-7c819cfea483 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:00.636840 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833ae84-b73d-4753-bf34-7c819cfea483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:00.643016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8833ae84-b73d-4753-bf34-7c819cfea483 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:00.700180 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1046/2074] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:30:00 2022] POST /identity/v3/projects => generated 361 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:00.701706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fca0da-013e-478c-bcf2-e9713a12e0c5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:00.715665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fca0da-013e-478c-bcf2-e9713a12e0c5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:00.796188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32da36d-43fc-4dc2-8eba-dfb0b7203532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:00.796904 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32da36d-43fc-4dc2-8eba-dfb0b7203532 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:00.797189 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32da36d-43fc-4dc2-8eba-dfb0b7203532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:00.797469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32da36d-43fc-4dc2-8eba-dfb0b7203532 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:00.798177 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32da36d-43fc-4dc2-8eba-dfb0b7203532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:00.800746 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32da36d-43fc-4dc2-8eba-dfb0b7203532 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:00.828267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1029/2075] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:30:00 2022] POST /identity/v3/users => generated 281 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:30:00.829578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833ae84-b73d-4753-bf34-7c819cfea483 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:00.840556 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833ae84-b73d-4753-bf34-7c819cfea483 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:00.848748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3e044-b25e-424e-93cf-7a8bb2d451a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:00.849429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3e044-b25e-424e-93cf-7a8bb2d451a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:00.849714 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3e044-b25e-424e-93cf-7a8bb2d451a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:00.850022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3e044-b25e-424e-93cf-7a8bb2d451a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:00.850725 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb3e044-b25e-424e-93cf-7a8bb2d451a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:00.852342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb3e044-b25e-424e-93cf-7a8bb2d451a7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:00.866394 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1047/2076] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:00.870214 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32da36d-43fc-4dc2-8eba-dfb0b7203532 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:00.884283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32da36d-43fc-4dc2-8eba-dfb0b7203532 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:00.889287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b40f9b-d405-44e2-b9d9-bad9acef7b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:00.889952 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b40f9b-d405-44e2-b9d9-bad9acef7b47 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:00.890233 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b40f9b-d405-44e2-b9d9-bad9acef7b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:00.890509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b40f9b-d405-44e2-b9d9-bad9acef7b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:00.891236 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b40f9b-d405-44e2-b9d9-bad9acef7b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:00.892867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b40f9b-d405-44e2-b9d9-bad9acef7b47 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:00.903622 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1030/2077] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:00.907608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3e044-b25e-424e-93cf-7a8bb2d451a7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:00.919224 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3e044-b25e-424e-93cf-7a8bb2d451a7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:00.924180 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca9469-7b30-48bf-b2d6-c4ae1da8bb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:00.925922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca9469-7b30-48bf-b2d6-c4ae1da8bb88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:00.926292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca9469-7b30-48bf-b2d6-c4ae1da8bb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:00.926553 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca9469-7b30-48bf-b2d6-c4ae1da8bb88 admin admin] PATH_INFO: `/v3/projects/d8cbf3ee9cee4f42b4b030e6287383ca/users/3e4c43f06da84f28b03a605a1470dfef/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:00.945906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca9469-7b30-48bf-b2d6-c4ae1da8bb88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8cbf3ee9cee4f42b4b030e6287383ca, user_id=3e4c43f06da84f28b03a605a1470dfef, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:00.951231 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca9469-7b30-48bf-b2d6-c4ae1da8bb88 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:00.970465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1048/2078] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:00 2022] PUT /identity/v3/projects/d8cbf3ee9cee4f42b4b030e6287383ca/users/3e4c43f06da84f28b03a605a1470dfef/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:00.981071 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ab23e-702b-4567-8cc5-57b1028eb5c6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:00.981367 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ab23e-702b-4567-8cc5-57b1028eb5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:00.981648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ab23e-702b-4567-8cc5-57b1028eb5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:01.012533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e79ab23e-702b-4567-8cc5-57b1028eb5c6 None None] MFA Rules not processed for user `3e4c43f06da84f28b03a605a1470dfef`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:01.013848 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e79ab23e-702b-4567-8cc5-57b1028eb5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:01.062875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:01.104836 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1031/2079] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:00 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:30:01.144742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca9469-7b30-48bf-b2d6-c4ae1da8bb88 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:01.146459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-13ca9469-7b30-48bf-b2d6-c4ae1da8bb88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:01.166622 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:01.243973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca9469-7b30-48bf-b2d6-c4ae1da8bb88 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:01.248699 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62bb04-f5bf-403f-820f-aff3da77687d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:01.249438 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62bb04-f5bf-403f-820f-aff3da77687d service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:01.249791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62bb04-f5bf-403f-820f-aff3da77687d service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:01.250139 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62bb04-f5bf-403f-820f-aff3da77687d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:01.261511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62bb04-f5bf-403f-820f-aff3da77687d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:01.263537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62bb04-f5bf-403f-820f-aff3da77687d service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:01.282068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1049/2080] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:01.297413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ab23e-702b-4567-8cc5-57b1028eb5c6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:01.387969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ab23e-702b-4567-8cc5-57b1028eb5c6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:01.392712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b0cf2-c690-4913-a49c-ac3d1aac023b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:01.393611 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b0cf2-c690-4913-a49c-ac3d1aac023b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:01.393897 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b0cf2-c690-4913-a49c-ac3d1aac023b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:01.394176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b0cf2-c690-4913-a49c-ac3d1aac023b admin admin] PATH_INFO: `/v3/users/3e4c43f06da84f28b03a605a1470dfef` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:01.395446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b0cf2-c690-4913-a49c-ac3d1aac023b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e4c43f06da84f28b03a605a1470dfef)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:01.397361 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b0cf2-c690-4913-a49c-ac3d1aac023b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:01.447557 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b4b0cf2-c690-4913-a49c-ac3d1aac023b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e4c43f06da84f28b03a605a1470dfef'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:01.448233 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b4b0cf2-c690-4913-a49c-ac3d1aac023b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e4c43f06da84f28b03a605a1470dfef'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:01.448845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b4b0cf2-c690-4913-a49c-ac3d1aac023b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e4c43f06da84f28b03a605a1470dfef'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:01.455991 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b4b0cf2-c690-4913-a49c-ac3d1aac023b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e4c43f06da84f28b03a605a1470dfef'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:01.471152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b4b0cf2-c690-4913-a49c-ac3d1aac023b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e4c43f06da84f28b03a605a1470dfef'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:01.492587 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1032/2081] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:01 2022] DELETE /identity/v3/users/3e4c43f06da84f28b03a605a1470dfef => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:01.497211 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dff67b-97ad-4fb0-b0b8-dc2ce26d7db1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:01.497615 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dff67b-97ad-4fb0-b0b8-dc2ce26d7db1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:01.497849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dff67b-97ad-4fb0-b0b8-dc2ce26d7db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:01.519184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-32dff67b-97ad-4fb0-b0b8-dc2ce26d7db1 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:01.520545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-32dff67b-97ad-4fb0-b0b8-dc2ce26d7db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:01.593983 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1050/2082] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:30:01.604443 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b0cf2-c690-4913-a49c-ac3d1aac023b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:01.607386 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9b4b0cf2-c690-4913-a49c-ac3d1aac023b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:01.691719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b0cf2-c690-4913-a49c-ac3d1aac023b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:01.697149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1254778-c692-4f81-abef-7e7873e8fea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:01.698085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1254778-c692-4f81-abef-7e7873e8fea5 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:01.698554 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1254778-c692-4f81-abef-7e7873e8fea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:01.698924 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1254778-c692-4f81-abef-7e7873e8fea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:01.711766 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1254778-c692-4f81-abef-7e7873e8fea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:01.713343 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1254778-c692-4f81-abef-7e7873e8fea5 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:01.718608 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:01.740289 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1033/2083] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:02.148392 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dff67b-97ad-4fb0-b0b8-dc2ce26d7db1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:02.156130 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-32dff67b-97ad-4fb0-b0b8-dc2ce26d7db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:02.170797 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dff67b-97ad-4fb0-b0b8-dc2ce26d7db1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:02.181150 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb521dc-b96b-4750-9a9b-ffa380653c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:02.182125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb521dc-b96b-4750-9a9b-ffa380653c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:02.182526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb521dc-b96b-4750-9a9b-ffa380653c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:02.183289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb521dc-b96b-4750-9a9b-ffa380653c90 admin admin] PATH_INFO: `/v3/projects/d8cbf3ee9cee4f42b4b030e6287383ca` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:02.184664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb521dc-b96b-4750-9a9b-ffa380653c90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8cbf3ee9cee4f42b4b030e6287383ca)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:02.186605 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb521dc-b96b-4750-9a9b-ffa380653c90 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:02.250672 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dcb521dc-b96b-4750-9a9b-ffa380653c90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8cbf3ee9cee4f42b4b030e6287383ca', 'request_id': 'req-dcb521dc-b96b-4750-9a9b-ffa380653c90'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:02.270125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dcb521dc-b96b-4750-9a9b-ffa380653c90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8cbf3ee9cee4f42b4b030e6287383ca', 'request_id': 'req-dcb521dc-b96b-4750-9a9b-ffa380653c90'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:02.278350 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dcb521dc-b96b-4750-9a9b-ffa380653c90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8cbf3ee9cee4f42b4b030e6287383ca', 'request_id': 'req-dcb521dc-b96b-4750-9a9b-ffa380653c90'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:02.293268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dcb521dc-b96b-4750-9a9b-ffa380653c90 admin admin] The token cache is being invalidate because project d8cbf3ee9cee4f42b4b030e6287383ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:02.293568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dcb521dc-b96b-4750-9a9b-ffa380653c90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:02.294238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dcb521dc-b96b-4750-9a9b-ffa380653c90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:02.296890 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1051/2084] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:02 2022] DELETE /identity/v3/projects/d8cbf3ee9cee4f42b4b030e6287383ca => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:02.302756 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825db39-12c6-4617-b283-e13d2202f790 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:02.303034 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825db39-12c6-4617-b283-e13d2202f790 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:02.303317 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825db39-12c6-4617-b283-e13d2202f790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:02.324482 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7825db39-12c6-4617-b283-e13d2202f790 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:02.325715 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7825db39-12c6-4617-b283-e13d2202f790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:02.394247 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1034/2085] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:02.401799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d5064-7516-4ff1-9e59-4fb9f03bce4e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:02.402093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d5064-7516-4ff1-9e59-4fb9f03bce4e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:02.402371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d5064-7516-4ff1-9e59-4fb9f03bce4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:02.422646 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a64d5064-7516-4ff1-9e59-4fb9f03bce4e None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:02.423786 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a64d5064-7516-4ff1-9e59-4fb9f03bce4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:02.431919 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1052/2086] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:30:02.436439 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825db39-12c6-4617-b283-e13d2202f790 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:02.446069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825db39-12c6-4617-b283-e13d2202f790 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:02.450667 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6d5bd-fc00-42e5-97f6-24de50673602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:02.451329 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6d5bd-fc00-42e5-97f6-24de50673602 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:02.451614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6d5bd-fc00-42e5-97f6-24de50673602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:02.451980 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6d5bd-fc00-42e5-97f6-24de50673602 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:02.452899 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6d5bd-fc00-42e5-97f6-24de50673602 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:30:02.453309 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6d5bd-fc00-42e5-97f6-24de50673602 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:02.455090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6d5bd-fc00-42e5-97f6-24de50673602 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:02.467965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1035/2087] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:30:02.472857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d5064-7516-4ff1-9e59-4fb9f03bce4e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:02.482346 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d5064-7516-4ff1-9e59-4fb9f03bce4e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:02.487449 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d13516-c448-4a93-a764-2457b92428a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:02.488098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d13516-c448-4a93-a764-2457b92428a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:02.488382 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d13516-c448-4a93-a764-2457b92428a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:02.488701 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d13516-c448-4a93-a764-2457b92428a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:02.489453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d13516-c448-4a93-a764-2457b92428a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:02.491442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d13516-c448-4a93-a764-2457b92428a3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:02.522047 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1053/2088] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:30:02 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:30:02.526230 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6d5bd-fc00-42e5-97f6-24de50673602 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:02.536409 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6d5bd-fc00-42e5-97f6-24de50673602 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:02.593662 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12865a-308c-478a-803a-8221bc9fe84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:02.594322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12865a-308c-478a-803a-8221bc9fe84d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:02.594606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12865a-308c-478a-803a-8221bc9fe84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:02.594885 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12865a-308c-478a-803a-8221bc9fe84d admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:02.595589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12865a-308c-478a-803a-8221bc9fe84d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:02.597635 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12865a-308c-478a-803a-8221bc9fe84d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:02.618005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1036/2089] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:30:02 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:30:02.623575 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d13516-c448-4a93-a764-2457b92428a3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:02.634389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d13516-c448-4a93-a764-2457b92428a3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:02.642367 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68037a6-cbf0-4c28-8855-a958b90ce143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:02.643050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68037a6-cbf0-4c28-8855-a958b90ce143 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:02.643968 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68037a6-cbf0-4c28-8855-a958b90ce143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:02.644254 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68037a6-cbf0-4c28-8855-a958b90ce143 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:02.645036 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68037a6-cbf0-4c28-8855-a958b90ce143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:02.646674 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68037a6-cbf0-4c28-8855-a958b90ce143 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:02.649149 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:02.659242 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1054/2090] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:02.663185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12865a-308c-478a-803a-8221bc9fe84d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:02.673383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12865a-308c-478a-803a-8221bc9fe84d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:02.677973 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad02f4-91ee-4c46-bf58-f62b2c69fc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:02.678627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad02f4-91ee-4c46-bf58-f62b2c69fc0a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:02.678915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad02f4-91ee-4c46-bf58-f62b2c69fc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:02.679194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad02f4-91ee-4c46-bf58-f62b2c69fc0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:02.679921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad02f4-91ee-4c46-bf58-f62b2c69fc0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:02.681553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad02f4-91ee-4c46-bf58-f62b2c69fc0a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:02.692386 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1037/2091] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:02.696650 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68037a6-cbf0-4c28-8855-a958b90ce143 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:02.707335 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68037a6-cbf0-4c28-8855-a958b90ce143 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:02.711884 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f93c57-932a-43de-99c9-598856596c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:02.712785 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f93c57-932a-43de-99c9-598856596c3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:02.713093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f93c57-932a-43de-99c9-598856596c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:02.713368 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f93c57-932a-43de-99c9-598856596c3d admin admin] PATH_INFO: `/v3/projects/ee6203312784435b8cd1fdfede3fb597/users/8c9439cb644548b8b3b5fd5c610757b9/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:02.728263 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f93c57-932a-43de-99c9-598856596c3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee6203312784435b8cd1fdfede3fb597, user_id=8c9439cb644548b8b3b5fd5c610757b9, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:02.731258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f93c57-932a-43de-99c9-598856596c3d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:02.744090 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1055/2092] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:02 2022] PUT /identity/v3/projects/ee6203312784435b8cd1fdfede3fb597/users/8c9439cb644548b8b3b5fd5c610757b9/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:02.749220 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d77091-1d0d-470b-ad4c-5bc87b0a0fb9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:02.749545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d77091-1d0d-470b-ad4c-5bc87b0a0fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:02.749873 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d77091-1d0d-470b-ad4c-5bc87b0a0fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:02.769641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-49d77091-1d0d-470b-ad4c-5bc87b0a0fb9 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:02.770859 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-49d77091-1d0d-470b-ad4c-5bc87b0a0fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:02.835314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1038/2093] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:02.844912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f93c57-932a-43de-99c9-598856596c3d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:02.846597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a9f93c57-932a-43de-99c9-598856596c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:02.910572 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f93c57-932a-43de-99c9-598856596c3d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:02.914646 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662db76c-44f1-42d0-a3ca-eab1b5fe5b68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:02.915377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662db76c-44f1-42d0-a3ca-eab1b5fe5b68 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:02.915751 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662db76c-44f1-42d0-a3ca-eab1b5fe5b68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:02.916100 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662db76c-44f1-42d0-a3ca-eab1b5fe5b68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:02.926864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662db76c-44f1-42d0-a3ca-eab1b5fe5b68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:02.928315 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662db76c-44f1-42d0-a3ca-eab1b5fe5b68 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:02.939884 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1056/2094] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:04.259271 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19486e64-acea-4ad9-b937-69eda58cf8cb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:04.260206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19486e64-acea-4ad9-b937-69eda58cf8cb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:04.260688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19486e64-acea-4ad9-b937-69eda58cf8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:04.270501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:04.314261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-19486e64-acea-4ad9-b937-69eda58cf8cb None None] MFA Rules not processed for user `8c9439cb644548b8b3b5fd5c610757b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:04.315765 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-19486e64-acea-4ad9-b937-69eda58cf8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:04.426678 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1039/2095] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:04.459748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662db76c-44f1-42d0-a3ca-eab1b5fe5b68 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:04.461289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-662db76c-44f1-42d0-a3ca-eab1b5fe5b68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:04.627573 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662db76c-44f1-42d0-a3ca-eab1b5fe5b68 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:04.635934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0c085-62d4-47b4-a805-f92a2572ccaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:04.639322 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0c085-62d4-47b4-a805-f92a2572ccaf service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:04.639619 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0c085-62d4-47b4-a805-f92a2572ccaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:04.639966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0c085-62d4-47b4-a805-f92a2572ccaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:04.653251 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0c085-62d4-47b4-a805-f92a2572ccaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:04.655364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0c085-62d4-47b4-a805-f92a2572ccaf service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:04.680994 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1057/2096] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:06.113102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c106bce-076f-4acc-b269-29c9146a16a9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:06.113625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c106bce-076f-4acc-b269-29c9146a16a9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:06.113732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c106bce-076f-4acc-b269-29c9146a16a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:06.132187 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5c106bce-076f-4acc-b269-29c9146a16a9 None None] MFA Rules not processed for user `8c9439cb644548b8b3b5fd5c610757b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:06.133380 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c106bce-076f-4acc-b269-29c9146a16a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:06.140642 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1040/2097] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:08.261645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c42c2-3c7d-44a5-a597-5587d828b351 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:08.262128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c42c2-3c7d-44a5-a597-5587d828b351 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:08.262209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c42c2-3c7d-44a5-a597-5587d828b351 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:08.263556 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1058/2098] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 18 10:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:30:08.295722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e71974-56ec-4edb-899f-005a85fca03b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:08.296299 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e71974-56ec-4edb-899f-005a85fca03b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:08.297774 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e71974-56ec-4edb-899f-005a85fca03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:08.325578 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-50e71974-56ec-4edb-899f-005a85fca03b None None] MFA Rules not processed for user `d07165f554a445169b2c3f0d173c0bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:08.326389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c42c2-3c7d-44a5-a597-5587d828b351 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:08.328168 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3b0c42c2-3c7d-44a5-a597-5587d828b351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:08.329625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-50e71974-56ec-4edb-899f-005a85fca03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:08.357921 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c42c2-3c7d-44a5-a597-5587d828b351 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:08.364327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6ac05-85e1-4d17-9894-832b13f7b018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:08.365219 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6ac05-85e1-4d17-9894-832b13f7b018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:08.365693 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6ac05-85e1-4d17-9894-832b13f7b018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:08.365973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6ac05-85e1-4d17-9894-832b13f7b018 admin admin] PATH_INFO: `/v3/users/c0f993e44c624f65abae22540ce7dffc` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:08.367267 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd6ac05-85e1-4d17-9894-832b13f7b018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0f993e44c624f65abae22540ce7dffc)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:08.369435 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd6ac05-85e1-4d17-9894-832b13f7b018 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:08.428459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1041/2099] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 18 10:30:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:08.441143 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0bd6ac05-85e1-4d17-9894-832b13f7b018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0f993e44c624f65abae22540ce7dffc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:08.447055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e71974-56ec-4edb-899f-005a85fca03b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:08.448886 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-50e71974-56ec-4edb-899f-005a85fca03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:08.462262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0bd6ac05-85e1-4d17-9894-832b13f7b018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0f993e44c624f65abae22540ce7dffc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:08.463295 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0bd6ac05-85e1-4d17-9894-832b13f7b018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0f993e44c624f65abae22540ce7dffc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:08.463932 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0bd6ac05-85e1-4d17-9894-832b13f7b018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0f993e44c624f65abae22540ce7dffc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:08.469661 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e71974-56ec-4edb-899f-005a85fca03b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:08.476474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0bd6ac05-85e1-4d17-9894-832b13f7b018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0f993e44c624f65abae22540ce7dffc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:08.479811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:08.483131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e34c6-7176-4bb6-bb10-29338c32d70b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd07165f554a445169b2c3f0d173c0bb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:08.483799 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e34c6-7176-4bb6-bb10-29338c32d70b service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:08.484143 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e34c6-7176-4bb6-bb10-29338c32d70b service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:08.484376 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e34c6-7176-4bb6-bb10-29338c32d70b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:08.490251 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c29e34c6-7176-4bb6-bb10-29338c32d70b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:08.497474 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1059/2100] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:08 2022] DELETE /identity/v3/users/c0f993e44c624f65abae22540ce7dffc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:30:08.504201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e34c6-7176-4bb6-bb10-29338c32d70b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:08.505962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e34c6-7176-4bb6-bb10-29338c32d70b service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:08.517754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1042/2101] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:08.767370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6ac05-85e1-4d17-9894-832b13f7b018 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:08.768967 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0bd6ac05-85e1-4d17-9894-832b13f7b018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:08.937717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6ac05-85e1-4d17-9894-832b13f7b018 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:08.944792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41841b0-7b92-4d60-912b-6f3279bfe856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:08.947522 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41841b0-7b92-4d60-912b-6f3279bfe856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:08.947872 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41841b0-7b92-4d60-912b-6f3279bfe856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:08.948221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41841b0-7b92-4d60-912b-6f3279bfe856 admin admin] PATH_INFO: `/v3/projects/daab623e57e54081b262d4a8fe124f42` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:08.950757 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41841b0-7b92-4d60-912b-6f3279bfe856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daab623e57e54081b262d4a8fe124f42)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:08.952845 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41841b0-7b92-4d60-912b-6f3279bfe856 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:09.062046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f41841b0-7b92-4d60-912b-6f3279bfe856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daab623e57e54081b262d4a8fe124f42', 'request_id': 'req-f41841b0-7b92-4d60-912b-6f3279bfe856'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:09.075735 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f41841b0-7b92-4d60-912b-6f3279bfe856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daab623e57e54081b262d4a8fe124f42', 'request_id': 'req-f41841b0-7b92-4d60-912b-6f3279bfe856'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:09.085050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f41841b0-7b92-4d60-912b-6f3279bfe856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daab623e57e54081b262d4a8fe124f42', 'request_id': 'req-f41841b0-7b92-4d60-912b-6f3279bfe856'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:09.116739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f41841b0-7b92-4d60-912b-6f3279bfe856 admin admin] The token cache is being invalidate because project daab623e57e54081b262d4a8fe124f42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:09.116950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f41841b0-7b92-4d60-912b-6f3279bfe856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:09.118831 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f41841b0-7b92-4d60-912b-6f3279bfe856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:09.123159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1060/2102] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:08 2022] DELETE /identity/v3/projects/daab623e57e54081b262d4a8fe124f42 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:09.128328 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea9c68-38ce-43d7-b100-9ffca356892a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:09.128931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea9c68-38ce-43d7-b100-9ffca356892a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:09.129279 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea9c68-38ce-43d7-b100-9ffca356892a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:09.152475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f1ea9c68-38ce-43d7-b100-9ffca356892a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:09.153807 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f1ea9c68-38ce-43d7-b100-9ffca356892a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:09.237194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1043/2103] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:09.246040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15554b-bff8-4882-8f18-260fc018b1dd None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:09.246684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15554b-bff8-4882-8f18-260fc018b1dd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:09.247141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15554b-bff8-4882-8f18-260fc018b1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:09.269745 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fa15554b-bff8-4882-8f18-260fc018b1dd None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:09.271713 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fa15554b-bff8-4882-8f18-260fc018b1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:09.280616 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1061/2104] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:09.284803 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea9c68-38ce-43d7-b100-9ffca356892a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:09.297297 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea9c68-38ce-43d7-b100-9ffca356892a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:09.307964 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffca5c-c308-491c-9235-597d4a3b4bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:09.308955 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffca5c-c308-491c-9235-597d4a3b4bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:09.309339 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffca5c-c308-491c-9235-597d4a3b4bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:09.309814 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffca5c-c308-491c-9235-597d4a3b4bc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:09.310654 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffca5c-c308-491c-9235-597d4a3b4bc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:30:09.311065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffca5c-c308-491c-9235-597d4a3b4bc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:09.312801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffca5c-c308-491c-9235-597d4a3b4bc3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:09.321517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:09.327880 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1044/2105] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:30:09.333047 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15554b-bff8-4882-8f18-260fc018b1dd None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:09.343694 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15554b-bff8-4882-8f18-260fc018b1dd None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:09.351731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9d6bd-4893-411e-9f65-61fa45c251a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:09.352354 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9d6bd-4893-411e-9f65-61fa45c251a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:09.352773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9d6bd-4893-411e-9f65-61fa45c251a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:09.353043 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9d6bd-4893-411e-9f65-61fa45c251a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:09.354038 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9d6bd-4893-411e-9f65-61fa45c251a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:09.356111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9d6bd-4893-411e-9f65-61fa45c251a5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:09.391211 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1062/2106] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:30:09 2022] POST /identity/v3/projects => generated 403 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:09.395845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffca5c-c308-491c-9235-597d4a3b4bc3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:09.412533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffca5c-c308-491c-9235-597d4a3b4bc3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:09.494112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f8da5-3704-42d6-bd5a-95b76d755809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:09.494889 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f8da5-3704-42d6-bd5a-95b76d755809 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:09.495180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f8da5-3704-42d6-bd5a-95b76d755809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:09.495458 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f8da5-3704-42d6-bd5a-95b76d755809 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:09.496157 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f8da5-3704-42d6-bd5a-95b76d755809 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:09.498133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f8da5-3704-42d6-bd5a-95b76d755809 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:09.523289 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1045/2107] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:30:09 2022] POST /identity/v3/users => generated 302 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:09.524743 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9d6bd-4893-411e-9f65-61fa45c251a5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:09.537286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9d6bd-4893-411e-9f65-61fa45c251a5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:09.542582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6ef8b-057e-40fa-adfe-83ea3954e762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:09.543255 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6ef8b-057e-40fa-adfe-83ea3954e762 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:09.543544 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6ef8b-057e-40fa-adfe-83ea3954e762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:09.543825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6ef8b-057e-40fa-adfe-83ea3954e762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:09.544626 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6ef8b-057e-40fa-adfe-83ea3954e762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:09.546246 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6ef8b-057e-40fa-adfe-83ea3954e762 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:09.558559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1063/2108] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:09.562577 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f8da5-3704-42d6-bd5a-95b76d755809 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:09.573088 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f8da5-3704-42d6-bd5a-95b76d755809 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:09.577829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b61c11-6b3f-4c5e-a948-788c51cea088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:09.578515 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b61c11-6b3f-4c5e-a948-788c51cea088 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:09.578788 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b61c11-6b3f-4c5e-a948-788c51cea088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:09.579066 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b61c11-6b3f-4c5e-a948-788c51cea088 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:09.579797 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b61c11-6b3f-4c5e-a948-788c51cea088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:09.581637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b61c11-6b3f-4c5e-a948-788c51cea088 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:09.595721 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1046/2109] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:09.597538 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6ef8b-057e-40fa-adfe-83ea3954e762 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:09.608130 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6ef8b-057e-40fa-adfe-83ea3954e762 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:09.613589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0d132-9866-44d1-9cde-b8e5a026e1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:09.614590 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0d132-9866-44d1-9cde-b8e5a026e1ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:09.614942 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0d132-9866-44d1-9cde-b8e5a026e1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:09.615290 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0d132-9866-44d1-9cde-b8e5a026e1ba admin admin] PATH_INFO: `/v3/projects/a6b9cf74829f49f39165fce79929ec54/users/475aa1580c4e42cda0862b460887d69a/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:09.631444 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0d132-9866-44d1-9cde-b8e5a026e1ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6b9cf74829f49f39165fce79929ec54, user_id=475aa1580c4e42cda0862b460887d69a, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:09.634169 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0d132-9866-44d1-9cde-b8e5a026e1ba admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:09.648541 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1064/2110] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:09 2022] PUT /identity/v3/projects/a6b9cf74829f49f39165fce79929ec54/users/475aa1580c4e42cda0862b460887d69a/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:09.653702 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce881ad-b65a-465d-86b3-1b920ed21bea None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:09.654303 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce881ad-b65a-465d-86b3-1b920ed21bea None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:09.654595 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce881ad-b65a-465d-86b3-1b920ed21bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:09.679834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cce881ad-b65a-465d-86b3-1b920ed21bea None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:09.681388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cce881ad-b65a-465d-86b3-1b920ed21bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:09.750015 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1047/2111] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:09.760005 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0d132-9866-44d1-9cde-b8e5a026e1ba admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:09.761792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1ca0d132-9866-44d1-9cde-b8e5a026e1ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:09.783043 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:09.825489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0d132-9866-44d1-9cde-b8e5a026e1ba admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:09.829301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d43e69-803c-4947-a0a3-7f45ed159b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:09.829958 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d43e69-803c-4947-a0a3-7f45ed159b1c service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:09.830245 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d43e69-803c-4947-a0a3-7f45ed159b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:09.831799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d43e69-803c-4947-a0a3-7f45ed159b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:09.842380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d43e69-803c-4947-a0a3-7f45ed159b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:09.843865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d43e69-803c-4947-a0a3-7f45ed159b1c service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:09.856517 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1065/2112] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:30:10.561653 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc416f-0c28-4b43-85bb-16c3f699f1f6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:10.562505 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc416f-0c28-4b43-85bb-16c3f699f1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:10.563484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc416f-0c28-4b43-85bb-16c3f699f1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:10.595596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4fcc416f-0c28-4b43-85bb-16c3f699f1f6 None None] MFA Rules not processed for user `475aa1580c4e42cda0862b460887d69a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:10.597327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4fcc416f-0c28-4b43-85bb-16c3f699f1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:10.679908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1048/2113] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:10.684570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d43e69-803c-4947-a0a3-7f45ed159b1c service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:10.696207 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d43e69-803c-4947-a0a3-7f45ed159b1c service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:10.702703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b228bf5-5300-4da6-ae84-5b05e8b17c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:10.703452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b228bf5-5300-4da6-ae84-5b05e8b17c68 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:10.703810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b228bf5-5300-4da6-ae84-5b05e8b17c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:10.704334 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b228bf5-5300-4da6-ae84-5b05e8b17c68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:10.705268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b228bf5-5300-4da6-ae84-5b05e8b17c68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:10.707575 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b228bf5-5300-4da6-ae84-5b05e8b17c68 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:10.744889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1066/2114] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:30:10 2022] POST /identity/v3/projects => generated 405 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:10.748184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc416f-0c28-4b43-85bb-16c3f699f1f6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:10.764945 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc416f-0c28-4b43-85bb-16c3f699f1f6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:10.769061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:10.829042 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414f672-77b8-40cf-85f8-3c4c2170d4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:10.829725 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414f672-77b8-40cf-85f8-3c4c2170d4b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:10.830015 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414f672-77b8-40cf-85f8-3c4c2170d4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:10.830297 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414f672-77b8-40cf-85f8-3c4c2170d4b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:10.831008 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414f672-77b8-40cf-85f8-3c4c2170d4b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:10.832992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414f672-77b8-40cf-85f8-3c4c2170d4b1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:10.854370 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1049/2115] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:30:10 2022] POST /identity/v3/users => generated 309 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:30:10.858276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b228bf5-5300-4da6-ae84-5b05e8b17c68 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:10.870068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b228bf5-5300-4da6-ae84-5b05e8b17c68 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:10.874760 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243544fe-53fb-4e00-b010-323047c97e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:10.875498 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243544fe-53fb-4e00-b010-323047c97e75 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:10.875849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243544fe-53fb-4e00-b010-323047c97e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:10.876194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243544fe-53fb-4e00-b010-323047c97e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:10.877052 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243544fe-53fb-4e00-b010-323047c97e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:10.878826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243544fe-53fb-4e00-b010-323047c97e75 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:10.890268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1067/2116] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:10.894320 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414f672-77b8-40cf-85f8-3c4c2170d4b1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:10.904862 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414f672-77b8-40cf-85f8-3c4c2170d4b1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:10.910829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80ec7c-abd8-464e-9d3e-1f7ea1fa0987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:10.911792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80ec7c-abd8-464e-9d3e-1f7ea1fa0987 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:10.912152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80ec7c-abd8-464e-9d3e-1f7ea1fa0987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:10.912532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80ec7c-abd8-464e-9d3e-1f7ea1fa0987 admin admin] PATH_INFO: `/v3/projects/6f7bf294f6f54a46a68c508ef50faab2/users/9347e142bb65408da2d0de53d629c7d5/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:10.928591 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80ec7c-abd8-464e-9d3e-1f7ea1fa0987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f7bf294f6f54a46a68c508ef50faab2, user_id=9347e142bb65408da2d0de53d629c7d5, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:10.931403 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80ec7c-abd8-464e-9d3e-1f7ea1fa0987 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:10.944384 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1050/2117] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:10 2022] PUT /identity/v3/projects/6f7bf294f6f54a46a68c508ef50faab2/users/9347e142bb65408da2d0de53d629c7d5/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:30:10.948491 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243544fe-53fb-4e00-b010-323047c97e75 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:10.959330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243544fe-53fb-4e00-b010-323047c97e75 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:11.016271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ce9c0-7a83-4f01-84b6-6001c00aca3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:11.016967 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ce9c0-7a83-4f01-84b6-6001c00aca3b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:11.017253 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ce9c0-7a83-4f01-84b6-6001c00aca3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:11.017533 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ce9c0-7a83-4f01-84b6-6001c00aca3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:11.018262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ce9c0-7a83-4f01-84b6-6001c00aca3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:11.019889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ce9c0-7a83-4f01-84b6-6001c00aca3b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:11.029718 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1068/2118] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:11.034481 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80ec7c-abd8-464e-9d3e-1f7ea1fa0987 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:11.045640 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80ec7c-abd8-464e-9d3e-1f7ea1fa0987 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:11.058534 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d37b88-d6e6-494f-a9d9-71960de3cf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:11.068498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d37b88-d6e6-494f-a9d9-71960de3cf59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:11.068907 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d37b88-d6e6-494f-a9d9-71960de3cf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:11.069261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d37b88-d6e6-494f-a9d9-71960de3cf59 admin admin] PATH_INFO: `/v3/projects/6f7bf294f6f54a46a68c508ef50faab2/users/9347e142bb65408da2d0de53d629c7d5/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:11.071923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d37b88-d6e6-494f-a9d9-71960de3cf59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f7bf294f6f54a46a68c508ef50faab2, user_id=9347e142bb65408da2d0de53d629c7d5, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:11.074951 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d37b88-d6e6-494f-a9d9-71960de3cf59 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:11.088098 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1051/2119] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:11 2022] PUT /identity/v3/projects/6f7bf294f6f54a46a68c508ef50faab2/users/9347e142bb65408da2d0de53d629c7d5/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:30:11.204040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1db21-91b6-4bb8-8966-c1a58b06433c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:11.204342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1db21-91b6-4bb8-8966-c1a58b06433c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:11.204664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1db21-91b6-4bb8-8966-c1a58b06433c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:11.206620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1069/2120] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 18 10:30:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:30:11.253249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695e250-d82c-4b9d-98e3-25e6506cebaa None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:11.258563 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695e250-d82c-4b9d-98e3-25e6506cebaa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:11.258969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695e250-d82c-4b9d-98e3-25e6506cebaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:11.290282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2695e250-d82c-4b9d-98e3-25e6506cebaa None None] MFA Rules not processed for user `d07165f554a445169b2c3f0d173c0bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:11.291964 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2695e250-d82c-4b9d-98e3-25e6506cebaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:11.446969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1052/2121] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 18 10:30:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:11.470020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1db21-91b6-4bb8-8966-c1a58b06433c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:11.472461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-99f1db21-91b6-4bb8-8966-c1a58b06433c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:11.490999 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1db21-91b6-4bb8-8966-c1a58b06433c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:11.504865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3d9d5-ccba-4e51-908d-fc3c03c2bb98 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd07165f554a445169b2c3f0d173c0bb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:11.505633 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3d9d5-ccba-4e51-908d-fc3c03c2bb98 service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:11.505985 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3d9d5-ccba-4e51-908d-fc3c03c2bb98 service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:11.506768 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3d9d5-ccba-4e51-908d-fc3c03c2bb98 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:11.528882 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3d9d5-ccba-4e51-908d-fc3c03c2bb98 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:11.530613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3d9d5-ccba-4e51-908d-fc3c03c2bb98 service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:11.557234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1070/2122] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:11.863102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d83d0-bf42-4bbf-8c57-34a6c0181b1f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:11.863636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d83d0-bf42-4bbf-8c57-34a6c0181b1f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:11.863636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d83d0-bf42-4bbf-8c57-34a6c0181b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:11.899183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-be3d83d0-bf42-4bbf-8c57-34a6c0181b1f None None] MFA Rules not processed for user `9347e142bb65408da2d0de53d629c7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:11.900608 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-be3d83d0-bf42-4bbf-8c57-34a6c0181b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:11.960752 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:12.008694 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1053/2123] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:11 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:13.161165 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3d9d5-ccba-4e51-908d-fc3c03c2bb98 service placement] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:13.163165 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dec3d9d5-ccba-4e51-908d-fc3c03c2bb98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:13.290301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3d9d5-ccba-4e51-908d-fc3c03c2bb98 service placement] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:13.302332 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc0eab-d908-4795-bf46-b298b78483ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:13.303503 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc0eab-d908-4795-bf46-b298b78483ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:13.303894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc0eab-d908-4795-bf46-b298b78483ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:13.304272 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc0eab-d908-4795-bf46-b298b78483ca admin admin] PATH_INFO: `/v3/users/0ba504cd4cd649dda79ba3a2112c45ef` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:13.305692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc0eab-d908-4795-bf46-b298b78483ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ba504cd4cd649dda79ba3a2112c45ef)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:13.307854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc0eab-d908-4795-bf46-b298b78483ca admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:13.322692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:13.388250 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-15dc0eab-d908-4795-bf46-b298b78483ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ba504cd4cd649dda79ba3a2112c45ef'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:13.416193 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-15dc0eab-d908-4795-bf46-b298b78483ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ba504cd4cd649dda79ba3a2112c45ef'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:13.417760 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-15dc0eab-d908-4795-bf46-b298b78483ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ba504cd4cd649dda79ba3a2112c45ef'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:13.418826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-15dc0eab-d908-4795-bf46-b298b78483ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ba504cd4cd649dda79ba3a2112c45ef'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:13.435383 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-15dc0eab-d908-4795-bf46-b298b78483ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ba504cd4cd649dda79ba3a2112c45ef'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:13.463423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1071/2124] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:13 2022] DELETE /identity/v3/users/0ba504cd4cd649dda79ba3a2112c45ef => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:13.602851 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d83d0-bf42-4bbf-8c57-34a6c0181b1f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:13.618044 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d83d0-bf42-4bbf-8c57-34a6c0181b1f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:13.690842 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36d72f-b8b9-43f3-824a-58b6858bc43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:13.691755 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36d72f-b8b9-43f3-824a-58b6858bc43b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:13.692037 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36d72f-b8b9-43f3-824a-58b6858bc43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:13.692317 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36d72f-b8b9-43f3-824a-58b6858bc43b admin admin] PATH_INFO: `/v3/users/475aa1580c4e42cda0862b460887d69a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:13.694105 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d36d72f-b8b9-43f3-824a-58b6858bc43b admin admin] RBAC: Authorizing `identity:delete_user(user_id=475aa1580c4e42cda0862b460887d69a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:13.696067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d36d72f-b8b9-43f3-824a-58b6858bc43b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:13.733124 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc0eab-d908-4795-bf46-b298b78483ca admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:13.734712 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-15dc0eab-d908-4795-bf46-b298b78483ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:13.754899 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc0eab-d908-4795-bf46-b298b78483ca admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:13.761125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e9509-ff28-42c9-a4e4-019a7d89e708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:13.762033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e9509-ff28-42c9-a4e4-019a7d89e708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:13.762319 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e9509-ff28-42c9-a4e4-019a7d89e708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:13.762594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e9509-ff28-42c9-a4e4-019a7d89e708 admin admin] PATH_INFO: `/v3/projects/1674901c25c74b63a834b11652e31149` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:13.763791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e9509-ff28-42c9-a4e4-019a7d89e708 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1674901c25c74b63a834b11652e31149)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:13.765794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e9509-ff28-42c9-a4e4-019a7d89e708 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:13.767145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8d36d72f-b8b9-43f3-824a-58b6858bc43b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '475aa1580c4e42cda0862b460887d69a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:13.768670 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8d36d72f-b8b9-43f3-824a-58b6858bc43b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '475aa1580c4e42cda0862b460887d69a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:13.769299 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8d36d72f-b8b9-43f3-824a-58b6858bc43b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '475aa1580c4e42cda0862b460887d69a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:13.778979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8d36d72f-b8b9-43f3-824a-58b6858bc43b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '475aa1580c4e42cda0862b460887d69a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:13.792103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8d36d72f-b8b9-43f3-824a-58b6858bc43b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '475aa1580c4e42cda0862b460887d69a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:13.811335 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1054/2125] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:13 2022] DELETE /identity/v3/users/475aa1580c4e42cda0862b460887d69a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:13.821095 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7d0e9509-ff28-42c9-a4e4-019a7d89e708 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1674901c25c74b63a834b11652e31149', 'request_id': 'req-7d0e9509-ff28-42c9-a4e4-019a7d89e708'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:13.827373 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7d0e9509-ff28-42c9-a4e4-019a7d89e708 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1674901c25c74b63a834b11652e31149', 'request_id': 'req-7d0e9509-ff28-42c9-a4e4-019a7d89e708'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:13.837147 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7d0e9509-ff28-42c9-a4e4-019a7d89e708 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1674901c25c74b63a834b11652e31149', 'request_id': 'req-7d0e9509-ff28-42c9-a4e4-019a7d89e708'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:13.863436 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7d0e9509-ff28-42c9-a4e4-019a7d89e708 admin admin] The token cache is being invalidate because project 1674901c25c74b63a834b11652e31149 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:13.863789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7d0e9509-ff28-42c9-a4e4-019a7d89e708 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:13.864481 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7d0e9509-ff28-42c9-a4e4-019a7d89e708 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:13.866419 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1072/2126] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:13 2022] DELETE /identity/v3/projects/1674901c25c74b63a834b11652e31149 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:13.878462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36d72f-b8b9-43f3-824a-58b6858bc43b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:13.880568 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8d36d72f-b8b9-43f3-824a-58b6858bc43b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:13.991039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36d72f-b8b9-43f3-824a-58b6858bc43b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:14.000683 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3aa692-8ff1-4fd4-b757-122c59287ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:14.002074 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3aa692-8ff1-4fd4-b757-122c59287ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:14.002741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3aa692-8ff1-4fd4-b757-122c59287ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:14.003148 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3aa692-8ff1-4fd4-b757-122c59287ea7 admin admin] PATH_INFO: `/v3/users/b2b483418bb946528522158286499bd4` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:14.004646 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3aa692-8ff1-4fd4-b757-122c59287ea7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2b483418bb946528522158286499bd4)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:14.006631 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3aa692-8ff1-4fd4-b757-122c59287ea7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:14.008732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:14.054163 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e9509-ff28-42c9-a4e4-019a7d89e708 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:14.055608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7d0e9509-ff28-42c9-a4e4-019a7d89e708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:14.075289 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6b3aa692-8ff1-4fd4-b757-122c59287ea7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2b483418bb946528522158286499bd4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.075958 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6b3aa692-8ff1-4fd4-b757-122c59287ea7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2b483418bb946528522158286499bd4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.076610 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6b3aa692-8ff1-4fd4-b757-122c59287ea7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2b483418bb946528522158286499bd4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.085374 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6b3aa692-8ff1-4fd4-b757-122c59287ea7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2b483418bb946528522158286499bd4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.104566 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6b3aa692-8ff1-4fd4-b757-122c59287ea7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2b483418bb946528522158286499bd4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.120064 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1055/2127] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:13 2022] DELETE /identity/v3/users/b2b483418bb946528522158286499bd4 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:30:14.145885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e9509-ff28-42c9-a4e4-019a7d89e708 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:14.162092 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:14.206122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22158404-8064-4ac3-9e75-3cd4baf9e3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:14.207312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22158404-8064-4ac3-9e75-3cd4baf9e3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:14.207602 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22158404-8064-4ac3-9e75-3cd4baf9e3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:14.207885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22158404-8064-4ac3-9e75-3cd4baf9e3e3 admin admin] PATH_INFO: `/v3/projects/a6b9cf74829f49f39165fce79929ec54` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:14.209118 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22158404-8064-4ac3-9e75-3cd4baf9e3e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6b9cf74829f49f39165fce79929ec54)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:14.211058 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22158404-8064-4ac3-9e75-3cd4baf9e3e3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:14.259745 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-22158404-8064-4ac3-9e75-3cd4baf9e3e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6b9cf74829f49f39165fce79929ec54', 'request_id': 'req-22158404-8064-4ac3-9e75-3cd4baf9e3e3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.265762 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-22158404-8064-4ac3-9e75-3cd4baf9e3e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6b9cf74829f49f39165fce79929ec54', 'request_id': 'req-22158404-8064-4ac3-9e75-3cd4baf9e3e3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.272623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-22158404-8064-4ac3-9e75-3cd4baf9e3e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6b9cf74829f49f39165fce79929ec54', 'request_id': 'req-22158404-8064-4ac3-9e75-3cd4baf9e3e3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.287395 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-22158404-8064-4ac3-9e75-3cd4baf9e3e3 admin admin] The token cache is being invalidate because project a6b9cf74829f49f39165fce79929ec54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:14.287782 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-22158404-8064-4ac3-9e75-3cd4baf9e3e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.288541 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-22158404-8064-4ac3-9e75-3cd4baf9e3e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.290448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1073/2128] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:14 2022] DELETE /identity/v3/projects/a6b9cf74829f49f39165fce79929ec54 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:30:14.294213 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3aa692-8ff1-4fd4-b757-122c59287ea7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:14.295760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6b3aa692-8ff1-4fd4-b757-122c59287ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:14.309563 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3aa692-8ff1-4fd4-b757-122c59287ea7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:14.357256 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22158404-8064-4ac3-9e75-3cd4baf9e3e3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:14.358745 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-22158404-8064-4ac3-9e75-3cd4baf9e3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:14.376370 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a40638-1462-4c32-b2ce-69755d1ba036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:14.377399 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a40638-1462-4c32-b2ce-69755d1ba036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:14.378186 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a40638-1462-4c32-b2ce-69755d1ba036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:14.378536 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a40638-1462-4c32-b2ce-69755d1ba036 admin admin] PATH_INFO: `/v3/users/9347e142bb65408da2d0de53d629c7d5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:14.379890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a40638-1462-4c32-b2ce-69755d1ba036 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9347e142bb65408da2d0de53d629c7d5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:14.383375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a40638-1462-4c32-b2ce-69755d1ba036 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:14.434612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-39a40638-1462-4c32-b2ce-69755d1ba036 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9347e142bb65408da2d0de53d629c7d5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.435192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-39a40638-1462-4c32-b2ce-69755d1ba036 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9347e142bb65408da2d0de53d629c7d5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.435378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22158404-8064-4ac3-9e75-3cd4baf9e3e3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:14.435800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-39a40638-1462-4c32-b2ce-69755d1ba036 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9347e142bb65408da2d0de53d629c7d5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.440694 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d92e4-9426-4ac8-aba3-d4e4f739a143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:14.441562 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d92e4-9426-4ac8-aba3-d4e4f739a143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:14.441850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d92e4-9426-4ac8-aba3-d4e4f739a143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:14.442151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d92e4-9426-4ac8-aba3-d4e4f739a143 admin admin] PATH_INFO: `/v3/projects/d33d7e2911114bd8b4364e6703069459` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:14.443339 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d92e4-9426-4ac8-aba3-d4e4f739a143 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d33d7e2911114bd8b4364e6703069459)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:14.443646 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-39a40638-1462-4c32-b2ce-69755d1ba036 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9347e142bb65408da2d0de53d629c7d5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.445298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d92e4-9426-4ac8-aba3-d4e4f739a143 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:14.455706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-39a40638-1462-4c32-b2ce-69755d1ba036 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9347e142bb65408da2d0de53d629c7d5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.464932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:14.473539 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1056/2129] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:14 2022] DELETE /identity/v3/users/9347e142bb65408da2d0de53d629c7d5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:14.499104 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-335d92e4-9426-4ac8-aba3-d4e4f739a143 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd33d7e2911114bd8b4364e6703069459', 'request_id': 'req-335d92e4-9426-4ac8-aba3-d4e4f739a143'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.501221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:14.509358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-335d92e4-9426-4ac8-aba3-d4e4f739a143 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd33d7e2911114bd8b4364e6703069459', 'request_id': 'req-335d92e4-9426-4ac8-aba3-d4e4f739a143'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.518199 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-335d92e4-9426-4ac8-aba3-d4e4f739a143 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd33d7e2911114bd8b4364e6703069459', 'request_id': 'req-335d92e4-9426-4ac8-aba3-d4e4f739a143'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.537829 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-335d92e4-9426-4ac8-aba3-d4e4f739a143 admin admin] The token cache is being invalidate because project d33d7e2911114bd8b4364e6703069459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:14.538139 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-335d92e4-9426-4ac8-aba3-d4e4f739a143 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.538784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-335d92e4-9426-4ac8-aba3-d4e4f739a143 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.540377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1074/2130] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:14 2022] DELETE /identity/v3/projects/d33d7e2911114bd8b4364e6703069459 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:30:14.545328 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f3b44-ddb4-4eb2-b77d-c7f0e22e914c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:14.545626 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f3b44-ddb4-4eb2-b77d-c7f0e22e914c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:14.545923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f3b44-ddb4-4eb2-b77d-c7f0e22e914c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:14.567849 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0a7f3b44-ddb4-4eb2-b77d-c7f0e22e914c None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:14.569288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0a7f3b44-ddb4-4eb2-b77d-c7f0e22e914c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:14.652675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1057/2131] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:14.657765 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16262527-ac4e-4f11-8aaa-f8060830264c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:14.658959 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16262527-ac4e-4f11-8aaa-f8060830264c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:14.659649 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16262527-ac4e-4f11-8aaa-f8060830264c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:14.687021 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-16262527-ac4e-4f11-8aaa-f8060830264c None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:14.688314 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-16262527-ac4e-4f11-8aaa-f8060830264c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:14.699441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1075/2132] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:14.702715 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f3b44-ddb4-4eb2-b77d-c7f0e22e914c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:14.714753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16262527-ac4e-4f11-8aaa-f8060830264c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:14.716778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f3b44-ddb4-4eb2-b77d-c7f0e22e914c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:14.718630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-16262527-ac4e-4f11-8aaa-f8060830264c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:14.734693 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16262527-ac4e-4f11-8aaa-f8060830264c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:14.740387 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c723aa0-0583-45a1-b215-850ebfbd8064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:14.741303 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c723aa0-0583-45a1-b215-850ebfbd8064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:14.741585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c723aa0-0583-45a1-b215-850ebfbd8064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:14.741863 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c723aa0-0583-45a1-b215-850ebfbd8064 admin admin] PATH_INFO: `/v3/projects/6f7bf294f6f54a46a68c508ef50faab2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:14.743110 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c723aa0-0583-45a1-b215-850ebfbd8064 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f7bf294f6f54a46a68c508ef50faab2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:14.745234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c723aa0-0583-45a1-b215-850ebfbd8064 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:14.753046 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b7474-6f7f-47c6-851c-9dd80b666731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:14.754432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b7474-6f7f-47c6-851c-9dd80b666731 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:14.754890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b7474-6f7f-47c6-851c-9dd80b666731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:14.755351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b7474-6f7f-47c6-851c-9dd80b666731 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:14.756280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b7474-6f7f-47c6-851c-9dd80b666731 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:30:14.756701 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b7474-6f7f-47c6-851c-9dd80b666731 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:14.758394 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b7474-6f7f-47c6-851c-9dd80b666731 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:14.772552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1058/2133] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:30:14.777290 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b7474-6f7f-47c6-851c-9dd80b666731 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:14.791041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b7474-6f7f-47c6-851c-9dd80b666731 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:14.799146 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3c723aa0-0583-45a1-b215-850ebfbd8064 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f7bf294f6f54a46a68c508ef50faab2', 'request_id': 'req-3c723aa0-0583-45a1-b215-850ebfbd8064'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.805512 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3c723aa0-0583-45a1-b215-850ebfbd8064 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f7bf294f6f54a46a68c508ef50faab2', 'request_id': 'req-3c723aa0-0583-45a1-b215-850ebfbd8064'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.812016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3c723aa0-0583-45a1-b215-850ebfbd8064 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f7bf294f6f54a46a68c508ef50faab2', 'request_id': 'req-3c723aa0-0583-45a1-b215-850ebfbd8064'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.825900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3c723aa0-0583-45a1-b215-850ebfbd8064 admin admin] The token cache is being invalidate because project 6f7bf294f6f54a46a68c508ef50faab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:14.826195 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3c723aa0-0583-45a1-b215-850ebfbd8064 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.826844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3c723aa0-0583-45a1-b215-850ebfbd8064 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:14.828556 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1076/2134] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:14 2022] DELETE /identity/v3/projects/6f7bf294f6f54a46a68c508ef50faab2 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:14.833342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21fd98-21fe-4643-a9fb-077bac4747a6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:14.833645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21fd98-21fe-4643-a9fb-077bac4747a6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:14.833932 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21fd98-21fe-4643-a9fb-077bac4747a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:14.851872 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8a21fd98-21fe-4643-a9fb-077bac4747a6 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:14.853212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8a21fd98-21fe-4643-a9fb-077bac4747a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:14.856275 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa4751-4d58-4c03-bfd3-0f5e59853e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:14.856976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa4751-4d58-4c03-bfd3-0f5e59853e81 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:14.857263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa4751-4d58-4c03-bfd3-0f5e59853e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:14.857540 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa4751-4d58-4c03-bfd3-0f5e59853e81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:14.858311 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa4751-4d58-4c03-bfd3-0f5e59853e81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:14.860423 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa4751-4d58-4c03-bfd3-0f5e59853e81 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:14.863834 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1077/2135] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:30:14.873650 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03848c-3356-4054-9b02-a6cff8fb2ff1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:14.873937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03848c-3356-4054-9b02-a6cff8fb2ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:14.874218 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03848c-3356-4054-9b02-a6cff8fb2ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:14.893454 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1059/2136] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:30:14 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:30:14.894039 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ef03848c-3356-4054-9b02-a6cff8fb2ff1 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:14.894650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa4751-4d58-4c03-bfd3-0f5e59853e81 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:14.895231 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ef03848c-3356-4054-9b02-a6cff8fb2ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:14.896388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b7fa4751-4d58-4c03-bfd3-0f5e59853e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:14.961966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1078/2137] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:30:14.966184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03848c-3356-4054-9b02-a6cff8fb2ff1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:14.968051 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa4751-4d58-4c03-bfd3-0f5e59853e81 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:14.973515 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f117b-cc6e-4054-ac36-4f1805c884c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:14.974194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f117b-cc6e-4054-ac36-4f1805c884c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:14.974477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f117b-cc6e-4054-ac36-4f1805c884c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:14.974753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f117b-cc6e-4054-ac36-4f1805c884c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:14.975475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f117b-cc6e-4054-ac36-4f1805c884c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:14.975743 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03848c-3356-4054-9b02-a6cff8fb2ff1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:14.985158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f117b-cc6e-4054-ac36-4f1805c884c7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:14.985305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900c937-3065-4efc-8862-478fe7250f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:14.985305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900c937-3065-4efc-8862-478fe7250f44 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:14.985305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900c937-3065-4efc-8862-478fe7250f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:14.985305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900c937-3065-4efc-8862-478fe7250f44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:14.985618 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900c937-3065-4efc-8862-478fe7250f44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:30:14.985618 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900c937-3065-4efc-8862-478fe7250f44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:14.985618 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900c937-3065-4efc-8862-478fe7250f44 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:14.994394 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:14.998670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1079/2138] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:30:15.000742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1060/2139] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:30:14 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:15.002849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900c937-3065-4efc-8862-478fe7250f44 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.004606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f117b-cc6e-4054-ac36-4f1805c884c7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.013301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900c937-3065-4efc-8862-478fe7250f44 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.015055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f117b-cc6e-4054-ac36-4f1805c884c7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.018712 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271075a-d5b9-47e0-b056-acb6647b84e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.019366 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271075a-d5b9-47e0-b056-acb6647b84e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.019653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271075a-d5b9-47e0-b056-acb6647b84e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.019933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271075a-d5b9-47e0-b056-acb6647b84e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.020188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbc5e1-236c-46d5-89ff-94ab99543cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.020849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271075a-d5b9-47e0-b056-acb6647b84e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.021459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbc5e1-236c-46d5-89ff-94ab99543cba admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.021763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbc5e1-236c-46d5-89ff-94ab99543cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.022027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbc5e1-236c-46d5-89ff-94ab99543cba admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.022760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbc5e1-236c-46d5-89ff-94ab99543cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.022867 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271075a-d5b9-47e0-b056-acb6647b84e2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.024340 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbc5e1-236c-46d5-89ff-94ab99543cba admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.035658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1061/2140] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:30:15.039677 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbc5e1-236c-46d5-89ff-94ab99543cba admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.049861 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbc5e1-236c-46d5-89ff-94ab99543cba admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.050646 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1080/2141] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:30:15 2022] POST /identity/v3/projects => generated 357 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:15.054616 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271075a-d5b9-47e0-b056-acb6647b84e2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.072487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:15.113402 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badde776-f58a-4455-bc0d-4f1c9253b5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.114192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badde776-f58a-4455-bc0d-4f1c9253b5cf admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.114550 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badde776-f58a-4455-bc0d-4f1c9253b5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.114840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badde776-f58a-4455-bc0d-4f1c9253b5cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.115583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badde776-f58a-4455-bc0d-4f1c9253b5cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.117228 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badde776-f58a-4455-bc0d-4f1c9253b5cf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.127560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1062/2142] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:15.131578 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badde776-f58a-4455-bc0d-4f1c9253b5cf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.135462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271075a-d5b9-47e0-b056-acb6647b84e2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.141505 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c1c01-359f-4f7e-8784-399a0b33983b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.142191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c1c01-359f-4f7e-8784-399a0b33983b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.142644 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c1c01-359f-4f7e-8784-399a0b33983b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.143552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c1c01-359f-4f7e-8784-399a0b33983b admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.144280 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c1c01-359f-4f7e-8784-399a0b33983b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.145826 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badde776-f58a-4455-bc0d-4f1c9253b5cf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.146275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c1c01-359f-4f7e-8784-399a0b33983b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.155979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a5efa-0eb9-4624-bb8d-eb07a46b12b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.155979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a5efa-0eb9-4624-bb8d-eb07a46b12b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.155979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a5efa-0eb9-4624-bb8d-eb07a46b12b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.155979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a5efa-0eb9-4624-bb8d-eb07a46b12b0 admin admin] PATH_INFO: `/v3/projects/993b4dce87824a75b93aef5d5c0aa1bd/users/30bc006680414d8e8e04331c1f9eaf4d/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.170290 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1081/2143] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:30:15 2022] POST /identity/v3/users => generated 279 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:15.174111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c1c01-359f-4f7e-8784-399a0b33983b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.181710 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a5efa-0eb9-4624-bb8d-eb07a46b12b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=993b4dce87824a75b93aef5d5c0aa1bd, user_id=30bc006680414d8e8e04331c1f9eaf4d, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.184917 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a5efa-0eb9-4624-bb8d-eb07a46b12b0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.185039 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c1c01-359f-4f7e-8784-399a0b33983b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.195255 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8079f61-ca1d-4592-b8e0-b0d4f7b5b9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.195951 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8079f61-ca1d-4592-b8e0-b0d4f7b5b9df admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.196307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8079f61-ca1d-4592-b8e0-b0d4f7b5b9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.196636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8079f61-ca1d-4592-b8e0-b0d4f7b5b9df admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.197455 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8079f61-ca1d-4592-b8e0-b0d4f7b5b9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.199131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8079f61-ca1d-4592-b8e0-b0d4f7b5b9df admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.204004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1063/2144] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:15 2022] PUT /identity/v3/projects/993b4dce87824a75b93aef5d5c0aa1bd/users/30bc006680414d8e8e04331c1f9eaf4d/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:30:15.212680 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1082/2145] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:30:15.214887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce8afc-43f5-4398-9982-3c13618e82aa None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.215183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce8afc-43f5-4398-9982-3c13618e82aa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.216296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8079f61-ca1d-4592-b8e0-b0d4f7b5b9df admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.216770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce8afc-43f5-4398-9982-3c13618e82aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.229338 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8079f61-ca1d-4592-b8e0-b0d4f7b5b9df admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.260143 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c2ce8afc-43f5-4398-9982-3c13618e82aa None None] MFA Rules not processed for user `30bc006680414d8e8e04331c1f9eaf4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:15.261484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c2ce8afc-43f5-4398-9982-3c13618e82aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:15.293393 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a5e97-495b-4ed3-988f-d54cb6d10a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.294093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a5e97-495b-4ed3-988f-d54cb6d10a1b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.294382 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a5e97-495b-4ed3-988f-d54cb6d10a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.294662 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a5e97-495b-4ed3-988f-d54cb6d10a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.295399 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a5e97-495b-4ed3-988f-d54cb6d10a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.297125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a5e97-495b-4ed3-988f-d54cb6d10a1b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.307713 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1083/2146] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:15.311890 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a5e97-495b-4ed3-988f-d54cb6d10a1b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.321758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a5e97-495b-4ed3-988f-d54cb6d10a1b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.327116 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ed818-8b86-48dd-a450-a34c1ed3d0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.330728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1064/2147] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:30:15.330878 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ed818-8b86-48dd-a450-a34c1ed3d0d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.330878 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ed818-8b86-48dd-a450-a34c1ed3d0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.330878 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ed818-8b86-48dd-a450-a34c1ed3d0d7 admin admin] PATH_INFO: `/v3/projects/1f6b50b861214381ae9aece1329dc6d8/users/856e8bb8a5f84e739cc3bc2d09d59aa6/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.331681 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce8afc-43f5-4398-9982-3c13618e82aa None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.341786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce8afc-43f5-4398-9982-3c13618e82aa None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.343982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ed818-8b86-48dd-a450-a34c1ed3d0d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f6b50b861214381ae9aece1329dc6d8, user_id=856e8bb8a5f84e739cc3bc2d09d59aa6, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.347259 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ed818-8b86-48dd-a450-a34c1ed3d0d7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.347601 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5bd22b-5437-4c82-9b23-472a7d962452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.348261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bd22b-5437-4c82-9b23-472a7d962452 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.348575 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bd22b-5437-4c82-9b23-472a7d962452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.348859 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bd22b-5437-4c82-9b23-472a7d962452 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.349556 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5bd22b-5437-4c82-9b23-472a7d962452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.351479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5bd22b-5437-4c82-9b23-472a7d962452 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.362941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1084/2148] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:15 2022] PUT /identity/v3/projects/1f6b50b861214381ae9aece1329dc6d8/users/856e8bb8a5f84e739cc3bc2d09d59aa6/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:30:15.374628 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e6a8d-2707-4313-81ac-24bd3c94ed13 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.374942 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e6a8d-2707-4313-81ac-24bd3c94ed13 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.375219 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e6a8d-2707-4313-81ac-24bd3c94ed13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.379295 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1065/2149] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:30:15 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:30:15.386882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5bd22b-5437-4c82-9b23-472a7d962452 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.403710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-674e6a8d-2707-4313-81ac-24bd3c94ed13 None None] MFA Rules not processed for user `856e8bb8a5f84e739cc3bc2d09d59aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:15.405027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-674e6a8d-2707-4313-81ac-24bd3c94ed13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:15.420382 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:15.453293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5bd22b-5437-4c82-9b23-472a7d962452 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.458594 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7eb01-e66e-4cd4-a946-208b6466b925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.459278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7eb01-e66e-4cd4-a946-208b6466b925 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.459568 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7eb01-e66e-4cd4-a946-208b6466b925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.459843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7eb01-e66e-4cd4-a946-208b6466b925 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.460575 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7eb01-e66e-4cd4-a946-208b6466b925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.460702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:15.462885 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7eb01-e66e-4cd4-a946-208b6466b925 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.472659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1085/2150] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:15 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:30:15.475779 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e6a8d-2707-4313-81ac-24bd3c94ed13 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.487502 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1066/2151] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:30:15 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:30:15.487692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e6a8d-2707-4313-81ac-24bd3c94ed13 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.488596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7eb01-e66e-4cd4-a946-208b6466b925 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.494853 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3efdff-845a-4e02-9a70-35d73c93152d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.495522 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3efdff-845a-4e02-9a70-35d73c93152d admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.495814 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3efdff-845a-4e02-9a70-35d73c93152d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.496094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3efdff-845a-4e02-9a70-35d73c93152d admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.496833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3efdff-845a-4e02-9a70-35d73c93152d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.498857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3efdff-845a-4e02-9a70-35d73c93152d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.506937 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7eb01-e66e-4cd4-a946-208b6466b925 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.512102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036fa9dc-8fce-4f26-b73a-b2b7fc74870b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.512810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fa9dc-8fce-4f26-b73a-b2b7fc74870b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.513105 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fa9dc-8fce-4f26-b73a-b2b7fc74870b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.513376 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fa9dc-8fce-4f26-b73a-b2b7fc74870b admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.514119 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036fa9dc-8fce-4f26-b73a-b2b7fc74870b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.515753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036fa9dc-8fce-4f26-b73a-b2b7fc74870b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.529024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1086/2152] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:30:15 2022] POST /identity/v3/projects => generated 355 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:30:15.529273 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1067/2153] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:15.530380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3efdff-845a-4e02-9a70-35d73c93152d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.530517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036fa9dc-8fce-4f26-b73a-b2b7fc74870b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.540797 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036fa9dc-8fce-4f26-b73a-b2b7fc74870b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.595301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3efdff-845a-4e02-9a70-35d73c93152d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.599279 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba4e67-1933-45f4-95ae-546c5a976245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.599944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba4e67-1933-45f4-95ae-546c5a976245 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.600228 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba4e67-1933-45f4-95ae-546c5a976245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.600428 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b7b64-5331-4ac0-82f2-587bb5dfa23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.600670 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba4e67-1933-45f4-95ae-546c5a976245 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.601218 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b7b64-5331-4ac0-82f2-587bb5dfa23f admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.601400 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba4e67-1933-45f4-95ae-546c5a976245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.601495 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b7b64-5331-4ac0-82f2-587bb5dfa23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.601769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b7b64-5331-4ac0-82f2-587bb5dfa23f admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.602461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b7b64-5331-4ac0-82f2-587bb5dfa23f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.603010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba4e67-1933-45f4-95ae-546c5a976245 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.604650 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b7b64-5331-4ac0-82f2-587bb5dfa23f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.613718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1068/2154] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:15.618994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba4e67-1933-45f4-95ae-546c5a976245 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.625331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1087/2155] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:30:15 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:15.629040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b7b64-5331-4ac0-82f2-587bb5dfa23f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.629159 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba4e67-1933-45f4-95ae-546c5a976245 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.633921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf37b77-5d4c-42aa-aaf0-54ad2baff6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.634685 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf37b77-5d4c-42aa-aaf0-54ad2baff6aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.634971 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf37b77-5d4c-42aa-aaf0-54ad2baff6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.635245 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf37b77-5d4c-42aa-aaf0-54ad2baff6aa admin admin] PATH_INFO: `/v3/projects/7b635ae6e3ac4bb6b68b37ec33596c7a/users/3bc8e9551ae645bfb38a87706ea5bcfb/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.638581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b7b64-5331-4ac0-82f2-587bb5dfa23f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.643821 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb1acb-6344-472e-8bf7-d075ddfee1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.644624 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb1acb-6344-472e-8bf7-d075ddfee1a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.644944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb1acb-6344-472e-8bf7-d075ddfee1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.645226 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb1acb-6344-472e-8bf7-d075ddfee1a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.645966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb1acb-6344-472e-8bf7-d075ddfee1a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.647612 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb1acb-6344-472e-8bf7-d075ddfee1a4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.650311 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf37b77-5d4c-42aa-aaf0-54ad2baff6aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b635ae6e3ac4bb6b68b37ec33596c7a, user_id=3bc8e9551ae645bfb38a87706ea5bcfb, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.653014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf37b77-5d4c-42aa-aaf0-54ad2baff6aa admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.662456 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1088/2156] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:30:15.663492 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb1acb-6344-472e-8bf7-d075ddfee1a4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.667030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1069/2157] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:15 2022] PUT /identity/v3/projects/7b635ae6e3ac4bb6b68b37ec33596c7a/users/3bc8e9551ae645bfb38a87706ea5bcfb/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:30:15.673685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb1acb-6344-472e-8bf7-d075ddfee1a4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.677226 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29c44c-56f0-4f1d-bdb2-502b8c726e71 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.677539 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29c44c-56f0-4f1d-bdb2-502b8c726e71 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.678000 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29c44c-56f0-4f1d-bdb2-502b8c726e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.705874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5c29c44c-56f0-4f1d-bdb2-502b8c726e71 None None] MFA Rules not processed for user `3bc8e9551ae645bfb38a87706ea5bcfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:15.707020 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c29c44c-56f0-4f1d-bdb2-502b8c726e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:15.732420 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a03c0-ee7f-443c-9ae6-155db79121d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.733458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a03c0-ee7f-443c-9ae6-155db79121d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.733752 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a03c0-ee7f-443c-9ae6-155db79121d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.734040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a03c0-ee7f-443c-9ae6-155db79121d9 admin admin] PATH_INFO: `/v3/projects/c664ad8c4d4949e38c439952013554aa/users/c88ac6fc28104157a462c1a7a8c3b5f0/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.752590 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a03c0-ee7f-443c-9ae6-155db79121d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c664ad8c4d4949e38c439952013554aa, user_id=c88ac6fc28104157a462c1a7a8c3b5f0, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.755321 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a03c0-ee7f-443c-9ae6-155db79121d9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.771226 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1089/2158] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:15 2022] PUT /identity/v3/projects/c664ad8c4d4949e38c439952013554aa/users/c88ac6fc28104157a462c1a7a8c3b5f0/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:30:15.775946 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a03c0-ee7f-443c-9ae6-155db79121d9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.786257 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1070/2159] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:30:15.790457 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29c44c-56f0-4f1d-bdb2-502b8c726e71 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.801542 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29c44c-56f0-4f1d-bdb2-502b8c726e71 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.806485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:15.824575 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:15.864874 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a03c0-ee7f-443c-9ae6-155db79121d9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.874438 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b501067-33ac-4a0d-b4f9-5b37da59cf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.876042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b501067-33ac-4a0d-b4f9-5b37da59cf09 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.876436 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b501067-33ac-4a0d-b4f9-5b37da59cf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.876755 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b501067-33ac-4a0d-b4f9-5b37da59cf09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.878699 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b501067-33ac-4a0d-b4f9-5b37da59cf09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.879027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5333588-f62a-4f7b-a49b-33feb3877a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.879799 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5333588-f62a-4f7b-a49b-33feb3877a00 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.880189 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5333588-f62a-4f7b-a49b-33feb3877a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.880467 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b501067-33ac-4a0d-b4f9-5b37da59cf09 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.880666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5333588-f62a-4f7b-a49b-33feb3877a00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.881469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5333588-f62a-4f7b-a49b-33feb3877a00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.883883 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5333588-f62a-4f7b-a49b-33feb3877a00 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.891734 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1090/2160] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:30:15.896591 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b501067-33ac-4a0d-b4f9-5b37da59cf09 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.909234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b501067-33ac-4a0d-b4f9-5b37da59cf09 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:15.914592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d0805-039a-48b1-8598-ef46532673bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:15.915588 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d0805-039a-48b1-8598-ef46532673bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.915876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d0805-039a-48b1-8598-ef46532673bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.916289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d0805-039a-48b1-8598-ef46532673bf admin admin] PATH_INFO: `/v3/projects/c664ad8c4d4949e38c439952013554aa/users/c88ac6fc28104157a462c1a7a8c3b5f0/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.918323 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d0805-039a-48b1-8598-ef46532673bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c664ad8c4d4949e38c439952013554aa, user_id=c88ac6fc28104157a462c1a7a8c3b5f0, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:15.920036 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1071/2161] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:30:15 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:30:15.921807 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d0805-039a-48b1-8598-ef46532673bf admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:15.924060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5333588-f62a-4f7b-a49b-33feb3877a00 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:15.950031 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1091/2162] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:15 2022] PUT /identity/v3/projects/c664ad8c4d4949e38c439952013554aa/users/c88ac6fc28104157a462c1a7a8c3b5f0/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:30:15.960259 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b491c-814c-4822-b27f-01715638ebf8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:15.960482 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b491c-814c-4822-b27f-01715638ebf8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:15.960826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b491c-814c-4822-b27f-01715638ebf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:15.989955 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-544b491c-814c-4822-b27f-01715638ebf8 None None] MFA Rules not processed for user `c88ac6fc28104157a462c1a7a8c3b5f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:15.991357 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-544b491c-814c-4822-b27f-01715638ebf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:16.027782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5333588-f62a-4f7b-a49b-33feb3877a00 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:16.071122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1092/2163] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:15 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:16.082717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b491c-814c-4822-b27f-01715638ebf8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:16.084089 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-544b491c-814c-4822-b27f-01715638ebf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:16.090635 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24a5eb-ea1e-4875-8c8e-c5c4418e6bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:16.091312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24a5eb-ea1e-4875-8c8e-c5c4418e6bec admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:16.091597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24a5eb-ea1e-4875-8c8e-c5c4418e6bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:16.091873 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24a5eb-ea1e-4875-8c8e-c5c4418e6bec admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:16.092597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24a5eb-ea1e-4875-8c8e-c5c4418e6bec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:16.094524 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24a5eb-ea1e-4875-8c8e-c5c4418e6bec admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:16.125863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1072/2164] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:30:15 2022] POST /identity/v3/users => generated 294 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:30:16.131454 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24a5eb-ea1e-4875-8c8e-c5c4418e6bec admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:16.142266 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24a5eb-ea1e-4875-8c8e-c5c4418e6bec admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:16.147201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf4f53-3793-40c9-88e1-c56f6e37ad78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:16.147369 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b491c-814c-4822-b27f-01715638ebf8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:16.148014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf4f53-3793-40c9-88e1-c56f6e37ad78 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:16.148363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf4f53-3793-40c9-88e1-c56f6e37ad78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:16.148744 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf4f53-3793-40c9-88e1-c56f6e37ad78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:16.149480 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf4f53-3793-40c9-88e1-c56f6e37ad78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:16.151701 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf4f53-3793-40c9-88e1-c56f6e37ad78 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:16.156795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98e85f-2d1e-4b11-8bba-4dedfce227b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:16.157535 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98e85f-2d1e-4b11-8bba-4dedfce227b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:16.157893 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98e85f-2d1e-4b11-8bba-4dedfce227b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:16.158286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98e85f-2d1e-4b11-8bba-4dedfce227b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:16.162872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1073/2165] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:16.166861 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf4f53-3793-40c9-88e1-c56f6e37ad78 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:16.169479 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d98e85f-2d1e-4b11-8bba-4dedfce227b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:16.171479 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d98e85f-2d1e-4b11-8bba-4dedfce227b7 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:16.177465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf4f53-3793-40c9-88e1-c56f6e37ad78 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:16.182301 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c451c-ea6f-41a6-9e78-1d4dff54e2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:16.182535 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1093/2166] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:16 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:30:16.183187 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c451c-ea6f-41a6-9e78-1d4dff54e2f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:16.183489 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c451c-ea6f-41a6-9e78-1d4dff54e2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:16.183769 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c451c-ea6f-41a6-9e78-1d4dff54e2f7 admin admin] PATH_INFO: `/v3/projects/ec2f751c482746efb59db292e1ea984a/users/d90939c758bc4007a12413c49d491836/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:16.199370 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c451c-ea6f-41a6-9e78-1d4dff54e2f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2f751c482746efb59db292e1ea984a, user_id=d90939c758bc4007a12413c49d491836, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:16.202359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c451c-ea6f-41a6-9e78-1d4dff54e2f7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:16.205531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:16.217641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1074/2167] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:16 2022] PUT /identity/v3/projects/ec2f751c482746efb59db292e1ea984a/users/d90939c758bc4007a12413c49d491836/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:30:16.222020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98e85f-2d1e-4b11-8bba-4dedfce227b7 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:16.233038 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98e85f-2d1e-4b11-8bba-4dedfce227b7 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:16.257873 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c451c-ea6f-41a6-9e78-1d4dff54e2f7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:16.283033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:16.300820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9f646-79a4-4724-bd4d-b45998a519fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:16.301600 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9f646-79a4-4724-bd4d-b45998a519fb admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:16.301950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9f646-79a4-4724-bd4d-b45998a519fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:16.302300 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9f646-79a4-4724-bd4d-b45998a519fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:16.303103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9f646-79a4-4724-bd4d-b45998a519fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:16.304810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9f646-79a4-4724-bd4d-b45998a519fb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:16.315224 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1094/2168] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:30:16.319066 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c451c-ea6f-41a6-9e78-1d4dff54e2f7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:16.319218 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9f646-79a4-4724-bd4d-b45998a519fb admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:16.323003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e4b00-1924-4959-b734-86e97abd40b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:16.324235 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e4b00-1924-4959-b734-86e97abd40b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:16.324235 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e4b00-1924-4959-b734-86e97abd40b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:16.328530 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e4b00-1924-4959-b734-86e97abd40b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:16.337548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9f646-79a4-4724-bd4d-b45998a519fb admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:16.341922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b553d0-62ce-422b-8a3a-cd6045cc4d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:16.342821 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b553d0-62ce-422b-8a3a-cd6045cc4d5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:16.343132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b553d0-62ce-422b-8a3a-cd6045cc4d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:16.343384 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b553d0-62ce-422b-8a3a-cd6045cc4d5b admin admin] PATH_INFO: `/v3/projects/ec2f751c482746efb59db292e1ea984a/users/d90939c758bc4007a12413c49d491836/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:16.345496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b553d0-62ce-422b-8a3a-cd6045cc4d5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2f751c482746efb59db292e1ea984a, user_id=d90939c758bc4007a12413c49d491836, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:16.348755 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b553d0-62ce-422b-8a3a-cd6045cc4d5b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:16.360835 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1095/2169] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:16 2022] PUT /identity/v3/projects/ec2f751c482746efb59db292e1ea984a/users/d90939c758bc4007a12413c49d491836/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:30:16.372385 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4f399-e532-4954-a659-4ea491901d9b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:16.373148 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4f399-e532-4954-a659-4ea491901d9b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:16.374070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4f399-e532-4954-a659-4ea491901d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:16.395295 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e4b00-1924-4959-b734-86e97abd40b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:16.396949 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e4b00-1924-4959-b734-86e97abd40b0 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:16.403951 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a3a4f399-e532-4954-a659-4ea491901d9b None None] MFA Rules not processed for user `d90939c758bc4007a12413c49d491836`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:16.405453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a3a4f399-e532-4954-a659-4ea491901d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:16.407318 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1075/2170] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:16 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:30:16.493350 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1096/2171] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:16 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:30:16.505764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e4b00-1924-4959-b734-86e97abd40b0 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:16.507807 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-158e4b00-1924-4959-b734-86e97abd40b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:16.524968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e4b00-1924-4959-b734-86e97abd40b0 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:16.619342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e01ab5-05c3-4f15-a20d-080ed4ae1a58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:16.620001 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e01ab5-05c3-4f15-a20d-080ed4ae1a58 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:16.620291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e01ab5-05c3-4f15-a20d-080ed4ae1a58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:16.620615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e01ab5-05c3-4f15-a20d-080ed4ae1a58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:16.689859 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e01ab5-05c3-4f15-a20d-080ed4ae1a58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:16.691324 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e01ab5-05c3-4f15-a20d-080ed4ae1a58 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:16.701070 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1076/2172] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:17.003053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653f315-26d3-40c1-882f-b797cf9ae2b0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:17.003498 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653f315-26d3-40c1-882f-b797cf9ae2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:17.003813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653f315-26d3-40c1-882f-b797cf9ae2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:17.030458 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5653f315-26d3-40c1-882f-b797cf9ae2b0 None None] MFA Rules not processed for user `bc233897770f47e5bd40fb03afabe49c`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:17.032178 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5653f315-26d3-40c1-882f-b797cf9ae2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:17.106401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1097/2173] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:18.292832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e01ab5-05c3-4f15-a20d-080ed4ae1a58 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:18.299904 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:18.304539 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e01ab5-05c3-4f15-a20d-080ed4ae1a58 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:18.310510 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116c379-6055-48b0-be36-859e6af545cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:18.311492 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116c379-6055-48b0-be36-859e6af545cb service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:18.311981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116c379-6055-48b0-be36-859e6af545cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:18.312530 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116c379-6055-48b0-be36-859e6af545cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:18.404807 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116c379-6055-48b0-be36-859e6af545cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:18.406278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116c379-6055-48b0-be36-859e6af545cb service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:18.427564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1077/2174] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:19.178463 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653f315-26d3-40c1-882f-b797cf9ae2b0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:19.188981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653f315-26d3-40c1-882f-b797cf9ae2b0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:19.194364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10f509-e9d8-41a0-87bc-ea2c810c845a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:19.195115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10f509-e9d8-41a0-87bc-ea2c810c845a service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:19.195412 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10f509-e9d8-41a0-87bc-ea2c810c845a service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:19.195696 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10f509-e9d8-41a0-87bc-ea2c810c845a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:19.207057 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10f509-e9d8-41a0-87bc-ea2c810c845a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:19.208704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10f509-e9d8-41a0-87bc-ea2c810c845a service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:19.221258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1098/2175] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:25.368415 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116c379-6055-48b0-be36-859e6af545cb service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:25.434149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116c379-6055-48b0-be36-859e6af545cb service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:25.438723 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be4755-9bd3-4446-9347-71fc0e6045c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:25.439612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be4755-9bd3-4446-9347-71fc0e6045c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:25.439879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be4755-9bd3-4446-9347-71fc0e6045c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:25.440159 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be4755-9bd3-4446-9347-71fc0e6045c2 admin admin] PATH_INFO: `/v3/users/30bc006680414d8e8e04331c1f9eaf4d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:25.441386 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be4755-9bd3-4446-9347-71fc0e6045c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30bc006680414d8e8e04331c1f9eaf4d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:25.443366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be4755-9bd3-4446-9347-71fc0e6045c2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:25.492860 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-80be4755-9bd3-4446-9347-71fc0e6045c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30bc006680414d8e8e04331c1f9eaf4d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:25.493512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-80be4755-9bd3-4446-9347-71fc0e6045c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30bc006680414d8e8e04331c1f9eaf4d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:25.494088 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-80be4755-9bd3-4446-9347-71fc0e6045c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30bc006680414d8e8e04331c1f9eaf4d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:25.500948 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-80be4755-9bd3-4446-9347-71fc0e6045c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30bc006680414d8e8e04331c1f9eaf4d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:25.512690 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-80be4755-9bd3-4446-9347-71fc0e6045c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30bc006680414d8e8e04331c1f9eaf4d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:25.525814 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1078/2176] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:25 2022] DELETE /identity/v3/users/30bc006680414d8e8e04331c1f9eaf4d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:25.530452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c37ca-4091-4d09-9b0b-a03673dca4c9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:25.530912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c37ca-4091-4d09-9b0b-a03673dca4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:25.531085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c37ca-4091-4d09-9b0b-a03673dca4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:25.549009 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-cc6c37ca-4091-4d09-9b0b-a03673dca4c9 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:25.550188 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cc6c37ca-4091-4d09-9b0b-a03673dca4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:25.610461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:25.641433 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1099/2177] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:25.652252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be4755-9bd3-4446-9347-71fc0e6045c2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:25.654417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-80be4755-9bd3-4446-9347-71fc0e6045c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:25.693722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:25.736271 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be4755-9bd3-4446-9347-71fc0e6045c2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:25.740749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443594e2-221b-4dd8-90fa-009602b63ff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:25.741765 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443594e2-221b-4dd8-90fa-009602b63ff4 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:25.742168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443594e2-221b-4dd8-90fa-009602b63ff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:25.742543 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443594e2-221b-4dd8-90fa-009602b63ff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:25.754736 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443594e2-221b-4dd8-90fa-009602b63ff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:25.756450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443594e2-221b-4dd8-90fa-009602b63ff4 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:25.769562 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1079/2178] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:26.137784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c37ca-4091-4d09-9b0b-a03673dca4c9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:26.139310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cc6c37ca-4091-4d09-9b0b-a03673dca4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:26.160708 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c37ca-4091-4d09-9b0b-a03673dca4c9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:26.165352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c8573-1542-48e6-b91a-4d8e69b59879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:26.166225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c8573-1542-48e6-b91a-4d8e69b59879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:26.166511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c8573-1542-48e6-b91a-4d8e69b59879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:26.166789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c8573-1542-48e6-b91a-4d8e69b59879 admin admin] PATH_INFO: `/v3/projects/993b4dce87824a75b93aef5d5c0aa1bd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:26.167983 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c8573-1542-48e6-b91a-4d8e69b59879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=993b4dce87824a75b93aef5d5c0aa1bd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:26.169957 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c8573-1542-48e6-b91a-4d8e69b59879 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:26.218572 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f55c8573-1542-48e6-b91a-4d8e69b59879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '993b4dce87824a75b93aef5d5c0aa1bd', 'request_id': 'req-f55c8573-1542-48e6-b91a-4d8e69b59879'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:26.224591 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f55c8573-1542-48e6-b91a-4d8e69b59879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '993b4dce87824a75b93aef5d5c0aa1bd', 'request_id': 'req-f55c8573-1542-48e6-b91a-4d8e69b59879'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:26.230968 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f55c8573-1542-48e6-b91a-4d8e69b59879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '993b4dce87824a75b93aef5d5c0aa1bd', 'request_id': 'req-f55c8573-1542-48e6-b91a-4d8e69b59879'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:26.243282 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f55c8573-1542-48e6-b91a-4d8e69b59879 admin admin] The token cache is being invalidate because project 993b4dce87824a75b93aef5d5c0aa1bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:26.243570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f55c8573-1542-48e6-b91a-4d8e69b59879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:26.244197 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f55c8573-1542-48e6-b91a-4d8e69b59879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:26.245774 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1100/2179] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:26 2022] DELETE /identity/v3/projects/993b4dce87824a75b93aef5d5c0aa1bd => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:26.249920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443594e2-221b-4dd8-90fa-009602b63ff4 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:26.259411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443594e2-221b-4dd8-90fa-009602b63ff4 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:26.315454 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6985fee-3947-4740-aba3-8c0e011e6567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:26.316393 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6985fee-3947-4740-aba3-8c0e011e6567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:26.316706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6985fee-3947-4740-aba3-8c0e011e6567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:26.316989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6985fee-3947-4740-aba3-8c0e011e6567 admin admin] PATH_INFO: `/v3/users/3bc8e9551ae645bfb38a87706ea5bcfb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:26.318175 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6985fee-3947-4740-aba3-8c0e011e6567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bc8e9551ae645bfb38a87706ea5bcfb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:26.320123 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6985fee-3947-4740-aba3-8c0e011e6567 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:26.368558 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b6985fee-3947-4740-aba3-8c0e011e6567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bc8e9551ae645bfb38a87706ea5bcfb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:26.369225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b6985fee-3947-4740-aba3-8c0e011e6567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bc8e9551ae645bfb38a87706ea5bcfb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:26.369838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b6985fee-3947-4740-aba3-8c0e011e6567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bc8e9551ae645bfb38a87706ea5bcfb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:26.377521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b6985fee-3947-4740-aba3-8c0e011e6567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bc8e9551ae645bfb38a87706ea5bcfb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:26.389496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b6985fee-3947-4740-aba3-8c0e011e6567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bc8e9551ae645bfb38a87706ea5bcfb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:26.403975 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1080/2180] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:26 2022] DELETE /identity/v3/users/3bc8e9551ae645bfb38a87706ea5bcfb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:26.703319 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c8573-1542-48e6-b91a-4d8e69b59879 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:26.704888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f55c8573-1542-48e6-b91a-4d8e69b59879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:26.719990 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c8573-1542-48e6-b91a-4d8e69b59879 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:26.778614 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17715c-fb32-4b23-9d5b-16c426e5906c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:26.779524 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17715c-fb32-4b23-9d5b-16c426e5906c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:26.779758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17715c-fb32-4b23-9d5b-16c426e5906c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:26.780031 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17715c-fb32-4b23-9d5b-16c426e5906c admin admin] PATH_INFO: `/v3/projects/7b635ae6e3ac4bb6b68b37ec33596c7a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:26.781327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17715c-fb32-4b23-9d5b-16c426e5906c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b635ae6e3ac4bb6b68b37ec33596c7a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:26.783312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17715c-fb32-4b23-9d5b-16c426e5906c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:26.833747 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ff17715c-fb32-4b23-9d5b-16c426e5906c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b635ae6e3ac4bb6b68b37ec33596c7a', 'request_id': 'req-ff17715c-fb32-4b23-9d5b-16c426e5906c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:26.839238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ff17715c-fb32-4b23-9d5b-16c426e5906c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b635ae6e3ac4bb6b68b37ec33596c7a', 'request_id': 'req-ff17715c-fb32-4b23-9d5b-16c426e5906c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:26.846312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ff17715c-fb32-4b23-9d5b-16c426e5906c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b635ae6e3ac4bb6b68b37ec33596c7a', 'request_id': 'req-ff17715c-fb32-4b23-9d5b-16c426e5906c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:26.857979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:26.862922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ff17715c-fb32-4b23-9d5b-16c426e5906c admin admin] The token cache is being invalidate because project 7b635ae6e3ac4bb6b68b37ec33596c7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:26.863548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ff17715c-fb32-4b23-9d5b-16c426e5906c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:26.867669 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ff17715c-fb32-4b23-9d5b-16c426e5906c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:26.870108 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1101/2181] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:26 2022] DELETE /identity/v3/projects/7b635ae6e3ac4bb6b68b37ec33596c7a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:26.874096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6985fee-3947-4740-aba3-8c0e011e6567 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:26.875488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b6985fee-3947-4740-aba3-8c0e011e6567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:26.889306 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6985fee-3947-4740-aba3-8c0e011e6567 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:26.931726 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:26.949229 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627ccf8-30f7-4276-a330-7a05f739dfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:26.950158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627ccf8-30f7-4276-a330-7a05f739dfbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:26.950442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627ccf8-30f7-4276-a330-7a05f739dfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:26.950720 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627ccf8-30f7-4276-a330-7a05f739dfbd admin admin] PATH_INFO: `/v3/users/d90939c758bc4007a12413c49d491836` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:26.951912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9627ccf8-30f7-4276-a330-7a05f739dfbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d90939c758bc4007a12413c49d491836)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:26.953883 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9627ccf8-30f7-4276-a330-7a05f739dfbd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:27.004414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9627ccf8-30f7-4276-a330-7a05f739dfbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd90939c758bc4007a12413c49d491836'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:27.005147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9627ccf8-30f7-4276-a330-7a05f739dfbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd90939c758bc4007a12413c49d491836'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:27.005765 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9627ccf8-30f7-4276-a330-7a05f739dfbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd90939c758bc4007a12413c49d491836'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:27.014110 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9627ccf8-30f7-4276-a330-7a05f739dfbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd90939c758bc4007a12413c49d491836'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:27.027529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9627ccf8-30f7-4276-a330-7a05f739dfbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd90939c758bc4007a12413c49d491836'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:27.043701 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1081/2182] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:26 2022] DELETE /identity/v3/users/d90939c758bc4007a12413c49d491836 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:27.443426 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17715c-fb32-4b23-9d5b-16c426e5906c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:27.445341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ff17715c-fb32-4b23-9d5b-16c426e5906c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:27.485313 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17715c-fb32-4b23-9d5b-16c426e5906c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:27.581171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1fdf3-4d85-4864-9eb0-a7b5705e3eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:27.582081 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1fdf3-4d85-4864-9eb0-a7b5705e3eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:27.582432 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1fdf3-4d85-4864-9eb0-a7b5705e3eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:27.582984 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1fdf3-4d85-4864-9eb0-a7b5705e3eed admin admin] PATH_INFO: `/v3/projects/ec2f751c482746efb59db292e1ea984a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:27.583982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1fdf3-4d85-4864-9eb0-a7b5705e3eed admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec2f751c482746efb59db292e1ea984a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:27.586053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1fdf3-4d85-4864-9eb0-a7b5705e3eed admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:27.667154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-55d1fdf3-4d85-4864-9eb0-a7b5705e3eed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec2f751c482746efb59db292e1ea984a', 'request_id': 'req-55d1fdf3-4d85-4864-9eb0-a7b5705e3eed'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:27.681312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-55d1fdf3-4d85-4864-9eb0-a7b5705e3eed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec2f751c482746efb59db292e1ea984a', 'request_id': 'req-55d1fdf3-4d85-4864-9eb0-a7b5705e3eed'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:27.695029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-55d1fdf3-4d85-4864-9eb0-a7b5705e3eed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec2f751c482746efb59db292e1ea984a', 'request_id': 'req-55d1fdf3-4d85-4864-9eb0-a7b5705e3eed'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:27.712604 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-55d1fdf3-4d85-4864-9eb0-a7b5705e3eed admin admin] The token cache is being invalidate because project ec2f751c482746efb59db292e1ea984a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:27.713261 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-55d1fdf3-4d85-4864-9eb0-a7b5705e3eed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:27.713745 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-55d1fdf3-4d85-4864-9eb0-a7b5705e3eed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:27.715510 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1102/2183] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:27 2022] DELETE /identity/v3/projects/ec2f751c482746efb59db292e1ea984a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:27.722129 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6a042-6450-45fd-8ea1-a4bef70df4f7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:27.722767 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6a042-6450-45fd-8ea1-a4bef70df4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:27.723232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6a042-6450-45fd-8ea1-a4bef70df4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:27.747312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c3e6a042-6450-45fd-8ea1-a4bef70df4f7 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:27.748913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c3e6a042-6450-45fd-8ea1-a4bef70df4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:27.814152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1082/2184] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:27.822029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56092b5-2e51-44fe-bd9d-90974582a94b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:27.822496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56092b5-2e51-44fe-bd9d-90974582a94b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:27.822822 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56092b5-2e51-44fe-bd9d-90974582a94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:27.844208 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b56092b5-2e51-44fe-bd9d-90974582a94b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:27.846461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b56092b5-2e51-44fe-bd9d-90974582a94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:27.855614 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1103/2185] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:30:27.860229 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6a042-6450-45fd-8ea1-a4bef70df4f7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:27.872705 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6a042-6450-45fd-8ea1-a4bef70df4f7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:27.877918 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b85ca-c9c7-4059-8d71-d837485322fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:27.878602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b85ca-c9c7-4059-8d71-d837485322fc admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:27.878894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b85ca-c9c7-4059-8d71-d837485322fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:27.879175 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b85ca-c9c7-4059-8d71-d837485322fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:27.880150 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b85ca-c9c7-4059-8d71-d837485322fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:30:27.880470 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b85ca-c9c7-4059-8d71-d837485322fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:27.882142 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b85ca-c9c7-4059-8d71-d837485322fc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:27.895625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1083/2186] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:30:27.900556 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56092b5-2e51-44fe-bd9d-90974582a94b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:27.912498 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56092b5-2e51-44fe-bd9d-90974582a94b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:27.917814 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8694da6-9d47-49b0-a733-12a8308d1cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:27.918511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8694da6-9d47-49b0-a733-12a8308d1cbf admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:27.918800 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8694da6-9d47-49b0-a733-12a8308d1cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:27.919077 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8694da6-9d47-49b0-a733-12a8308d1cbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:27.919786 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8694da6-9d47-49b0-a733-12a8308d1cbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:27.921747 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8694da6-9d47-49b0-a733-12a8308d1cbf admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:27.971641 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1104/2187] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:30:27 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:27.975983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b85ca-c9c7-4059-8d71-d837485322fc admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:27.988233 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b85ca-c9c7-4059-8d71-d837485322fc admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:28.073588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:28.104598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379b0a4-3770-4bd8-876b-55a93c320228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:28.105987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379b0a4-3770-4bd8-876b-55a93c320228 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:28.106373 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379b0a4-3770-4bd8-876b-55a93c320228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:28.106719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379b0a4-3770-4bd8-876b-55a93c320228 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:28.108145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379b0a4-3770-4bd8-876b-55a93c320228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:28.111089 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379b0a4-3770-4bd8-876b-55a93c320228 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:28.139616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1084/2188] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:30:27 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:30:28.141167 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8694da6-9d47-49b0-a733-12a8308d1cbf admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:28.161592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8694da6-9d47-49b0-a733-12a8308d1cbf admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:28.171584 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c9531-9232-4ad9-af97-4c6c4b3af039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:28.172531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c9531-9232-4ad9-af97-4c6c4b3af039 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:28.173254 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c9531-9232-4ad9-af97-4c6c4b3af039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:28.173732 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c9531-9232-4ad9-af97-4c6c4b3af039 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:28.175270 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c9531-9232-4ad9-af97-4c6c4b3af039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:28.177311 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c9531-9232-4ad9-af97-4c6c4b3af039 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:28.195988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1105/2189] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:28.197453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379b0a4-3770-4bd8-876b-55a93c320228 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:28.209112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379b0a4-3770-4bd8-876b-55a93c320228 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:28.214583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090798c-a249-433c-a15e-13c15ee79776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:28.215252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090798c-a249-433c-a15e-13c15ee79776 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:28.215935 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090798c-a249-433c-a15e-13c15ee79776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:28.215935 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090798c-a249-433c-a15e-13c15ee79776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:28.216873 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090798c-a249-433c-a15e-13c15ee79776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:28.219108 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090798c-a249-433c-a15e-13c15ee79776 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:28.231244 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1085/2190] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:30:28.235884 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c9531-9232-4ad9-af97-4c6c4b3af039 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:28.246128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c9531-9232-4ad9-af97-4c6c4b3af039 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:28.251504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496eb7c-15cf-4d26-a268-7294e600fab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:28.252988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496eb7c-15cf-4d26-a268-7294e600fab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:28.253263 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496eb7c-15cf-4d26-a268-7294e600fab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:28.253538 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496eb7c-15cf-4d26-a268-7294e600fab5 admin admin] PATH_INFO: `/v3/projects/2874c7f3748344f88a929205823b8131/users/261c58eaf53c45c999ad6fadaa91c921/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:28.271731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496eb7c-15cf-4d26-a268-7294e600fab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2874c7f3748344f88a929205823b8131, user_id=261c58eaf53c45c999ad6fadaa91c921, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:28.274979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496eb7c-15cf-4d26-a268-7294e600fab5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:28.289113 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1106/2191] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:28 2022] PUT /identity/v3/projects/2874c7f3748344f88a929205823b8131/users/261c58eaf53c45c999ad6fadaa91c921/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:28.300771 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a076d-a791-481f-a9bb-d5c010b593f8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:28.301423 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a076d-a791-481f-a9bb-d5c010b593f8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:28.301725 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a076d-a791-481f-a9bb-d5c010b593f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:28.331590 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ad6a076d-a791-481f-a9bb-d5c010b593f8 None None] MFA Rules not processed for user `261c58eaf53c45c999ad6fadaa91c921`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:28.332880 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ad6a076d-a791-481f-a9bb-d5c010b593f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:28.407341 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1086/2192] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:28.418853 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496eb7c-15cf-4d26-a268-7294e600fab5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:28.421484 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a496eb7c-15cf-4d26-a268-7294e600fab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:28.456127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:28.488620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496eb7c-15cf-4d26-a268-7294e600fab5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:28.493588 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb375a4-6135-4820-adc3-9c6d019e7b4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:28.494288 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb375a4-6135-4820-adc3-9c6d019e7b4a service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:28.494607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb375a4-6135-4820-adc3-9c6d019e7b4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:28.494915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb375a4-6135-4820-adc3-9c6d019e7b4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:28.505985 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb375a4-6135-4820-adc3-9c6d019e7b4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:28.507464 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb375a4-6135-4820-adc3-9c6d019e7b4a service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:28.519234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1107/2193] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:30.300634 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a076d-a791-481f-a9bb-d5c010b593f8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:30.302812 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ad6a076d-a791-481f-a9bb-d5c010b593f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:30.383089 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a076d-a791-481f-a9bb-d5c010b593f8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:30.387748 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75d22d-a726-4c8a-bdfa-d27b6cc25552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:30.388660 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75d22d-a726-4c8a-bdfa-d27b6cc25552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:30.388948 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75d22d-a726-4c8a-bdfa-d27b6cc25552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:30.389227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75d22d-a726-4c8a-bdfa-d27b6cc25552 admin admin] PATH_INFO: `/v3/users/f31b10a9ece749158fff645b7eb0f94a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:30.390443 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75d22d-a726-4c8a-bdfa-d27b6cc25552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f31b10a9ece749158fff645b7eb0f94a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:30.393026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75d22d-a726-4c8a-bdfa-d27b6cc25552 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:30.446149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb75d22d-a726-4c8a-bdfa-d27b6cc25552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f31b10a9ece749158fff645b7eb0f94a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:30.446809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb75d22d-a726-4c8a-bdfa-d27b6cc25552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f31b10a9ece749158fff645b7eb0f94a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:30.447411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb75d22d-a726-4c8a-bdfa-d27b6cc25552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f31b10a9ece749158fff645b7eb0f94a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:30.454669 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb75d22d-a726-4c8a-bdfa-d27b6cc25552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f31b10a9ece749158fff645b7eb0f94a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:30.460824 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:30.470702 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb75d22d-a726-4c8a-bdfa-d27b6cc25552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f31b10a9ece749158fff645b7eb0f94a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:30.484937 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1087/2194] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:30 2022] DELETE /identity/v3/users/f31b10a9ece749158fff645b7eb0f94a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:30.489598 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0ffcb-6ee2-4268-b0ee-ed5a15cb694f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:30.490108 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0ffcb-6ee2-4268-b0ee-ed5a15cb694f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:30.490221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0ffcb-6ee2-4268-b0ee-ed5a15cb694f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:30.508899 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-dbc0ffcb-6ee2-4268-b0ee-ed5a15cb694f None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:30.510124 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dbc0ffcb-6ee2-4268-b0ee-ed5a15cb694f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:30.590775 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1108/2195] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:30.600864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75d22d-a726-4c8a-bdfa-d27b6cc25552 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:30.602311 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fb75d22d-a726-4c8a-bdfa-d27b6cc25552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:30.661914 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75d22d-a726-4c8a-bdfa-d27b6cc25552 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:30.665576 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9672e-41e5-4158-9b8f-0711429dcd8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:30.666224 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9672e-41e5-4158-9b8f-0711429dcd8e service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:30.666508 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9672e-41e5-4158-9b8f-0711429dcd8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:30.666787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9672e-41e5-4158-9b8f-0711429dcd8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:30.676162 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9672e-41e5-4158-9b8f-0711429dcd8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:30.677627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9672e-41e5-4158-9b8f-0711429dcd8e service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:30.687486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1088/2196] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:30.980194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc0ffcb-6ee2-4268-b0ee-ed5a15cb694f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:30.981692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dbc0ffcb-6ee2-4268-b0ee-ed5a15cb694f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:30.994716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc0ffcb-6ee2-4268-b0ee-ed5a15cb694f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:30.999637 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb076f29-8f94-49b4-b7f6-a963c36caff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:31.000549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb076f29-8f94-49b4-b7f6-a963c36caff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:31.000833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb076f29-8f94-49b4-b7f6-a963c36caff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:31.001118 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb076f29-8f94-49b4-b7f6-a963c36caff9 admin admin] PATH_INFO: `/v3/projects/1752330828e74475a96d782a1aa0b3f6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:31.002919 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb076f29-8f94-49b4-b7f6-a963c36caff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1752330828e74475a96d782a1aa0b3f6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:31.004814 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb076f29-8f94-49b4-b7f6-a963c36caff9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:31.054056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cb076f29-8f94-49b4-b7f6-a963c36caff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1752330828e74475a96d782a1aa0b3f6', 'request_id': 'req-cb076f29-8f94-49b4-b7f6-a963c36caff9'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:31.059519 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cb076f29-8f94-49b4-b7f6-a963c36caff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1752330828e74475a96d782a1aa0b3f6', 'request_id': 'req-cb076f29-8f94-49b4-b7f6-a963c36caff9'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:31.065710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cb076f29-8f94-49b4-b7f6-a963c36caff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1752330828e74475a96d782a1aa0b3f6', 'request_id': 'req-cb076f29-8f94-49b4-b7f6-a963c36caff9'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:31.081794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cb076f29-8f94-49b4-b7f6-a963c36caff9 admin admin] The token cache is being invalidate because project 1752330828e74475a96d782a1aa0b3f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:31.082084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cb076f29-8f94-49b4-b7f6-a963c36caff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:31.082717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-cb076f29-8f94-49b4-b7f6-a963c36caff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:31.084275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1109/2197] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:30 2022] DELETE /identity/v3/projects/1752330828e74475a96d782a1aa0b3f6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:31.297128 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9672e-41e5-4158-9b8f-0711429dcd8e service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:31.298739 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-84d9672e-41e5-4158-9b8f-0711429dcd8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:31.311848 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9672e-41e5-4158-9b8f-0711429dcd8e service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:31.374055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a115b-84ec-425d-a1a1-7eb361b16ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:31.374944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a115b-84ec-425d-a1a1-7eb361b16ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:31.375220 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a115b-84ec-425d-a1a1-7eb361b16ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:31.375495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a115b-84ec-425d-a1a1-7eb361b16ae1 admin admin] PATH_INFO: `/v3/users/261c58eaf53c45c999ad6fadaa91c921` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:31.377273 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a115b-84ec-425d-a1a1-7eb361b16ae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=261c58eaf53c45c999ad6fadaa91c921)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:31.379313 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a115b-84ec-425d-a1a1-7eb361b16ae1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:31.433151 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6d6a115b-84ec-425d-a1a1-7eb361b16ae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '261c58eaf53c45c999ad6fadaa91c921'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:31.434304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6d6a115b-84ec-425d-a1a1-7eb361b16ae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '261c58eaf53c45c999ad6fadaa91c921'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:31.435005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6d6a115b-84ec-425d-a1a1-7eb361b16ae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '261c58eaf53c45c999ad6fadaa91c921'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:31.457161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6d6a115b-84ec-425d-a1a1-7eb361b16ae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '261c58eaf53c45c999ad6fadaa91c921'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:31.472443 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6d6a115b-84ec-425d-a1a1-7eb361b16ae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '261c58eaf53c45c999ad6fadaa91c921'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:31.487758 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:31.494710 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1089/2198] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:31 2022] DELETE /identity/v3/users/261c58eaf53c45c999ad6fadaa91c921 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:31.498749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa37fb1-f5af-4179-a908-e9991dabd234 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:31.499212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa37fb1-f5af-4179-a908-e9991dabd234 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:31.499310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa37fb1-f5af-4179-a908-e9991dabd234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:31.519930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5aa37fb1-f5af-4179-a908-e9991dabd234 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:31.521429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5aa37fb1-f5af-4179-a908-e9991dabd234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:31.579641 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:31.612766 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1110/2199] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:31.620163 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a115b-84ec-425d-a1a1-7eb361b16ae1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:31.621661 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6d6a115b-84ec-425d-a1a1-7eb361b16ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:31.685918 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a115b-84ec-425d-a1a1-7eb361b16ae1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:31.689619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c35f1-18eb-4b74-a9a2-5f5a2bfc745a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:31.690322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c35f1-18eb-4b74-a9a2-5f5a2bfc745a service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:31.690651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c35f1-18eb-4b74-a9a2-5f5a2bfc745a service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:31.690930 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c35f1-18eb-4b74-a9a2-5f5a2bfc745a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:31.701962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c35f1-18eb-4b74-a9a2-5f5a2bfc745a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:31.703699 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c35f1-18eb-4b74-a9a2-5f5a2bfc745a service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:31.714557 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1090/2200] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:31.986373 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa37fb1-f5af-4179-a908-e9991dabd234 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:31.987850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5aa37fb1-f5af-4179-a908-e9991dabd234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:32.001212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa37fb1-f5af-4179-a908-e9991dabd234 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:32.006409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd3f54-37e7-45d4-afb2-e98d207b2236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:32.007740 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd3f54-37e7-45d4-afb2-e98d207b2236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:32.007740 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd3f54-37e7-45d4-afb2-e98d207b2236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:32.007945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd3f54-37e7-45d4-afb2-e98d207b2236 admin admin] PATH_INFO: `/v3/projects/2874c7f3748344f88a929205823b8131` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:32.009179 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd3f54-37e7-45d4-afb2-e98d207b2236 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2874c7f3748344f88a929205823b8131)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:32.011197 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd3f54-37e7-45d4-afb2-e98d207b2236 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:32.061609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b4dd3f54-37e7-45d4-afb2-e98d207b2236 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2874c7f3748344f88a929205823b8131', 'request_id': 'req-b4dd3f54-37e7-45d4-afb2-e98d207b2236'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:32.067527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b4dd3f54-37e7-45d4-afb2-e98d207b2236 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2874c7f3748344f88a929205823b8131', 'request_id': 'req-b4dd3f54-37e7-45d4-afb2-e98d207b2236'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:32.073840 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b4dd3f54-37e7-45d4-afb2-e98d207b2236 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2874c7f3748344f88a929205823b8131', 'request_id': 'req-b4dd3f54-37e7-45d4-afb2-e98d207b2236'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:32.087681 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b4dd3f54-37e7-45d4-afb2-e98d207b2236 admin admin] The token cache is being invalidate because project 2874c7f3748344f88a929205823b8131 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:32.087970 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b4dd3f54-37e7-45d4-afb2-e98d207b2236 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:32.088617 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b4dd3f54-37e7-45d4-afb2-e98d207b2236 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:32.090191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1111/2201] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:31 2022] DELETE /identity/v3/projects/2874c7f3748344f88a929205823b8131 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:32.095033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df9bc3-9aab-42b0-b7c5-5e8a3493637b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:32.095328 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df9bc3-9aab-42b0-b7c5-5e8a3493637b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:32.095610 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df9bc3-9aab-42b0-b7c5-5e8a3493637b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:32.115994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f1df9bc3-9aab-42b0-b7c5-5e8a3493637b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:32.117233 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f1df9bc3-9aab-42b0-b7c5-5e8a3493637b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:32.181761 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1091/2202] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:30:32.190311 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab415b-0c5e-4bea-bf04-3c47ad32b5fb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:32.190604 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab415b-0c5e-4bea-bf04-3c47ad32b5fb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:32.190890 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab415b-0c5e-4bea-bf04-3c47ad32b5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:32.209166 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3aab415b-0c5e-4bea-bf04-3c47ad32b5fb None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:32.210403 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3aab415b-0c5e-4bea-bf04-3c47ad32b5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:32.219402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1112/2203] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:32.223768 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df9bc3-9aab-42b0-b7c5-5e8a3493637b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:32.237297 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df9bc3-9aab-42b0-b7c5-5e8a3493637b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:32.246923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82147578-b281-464d-a701-ae91d03c4a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:32.248037 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82147578-b281-464d-a701-ae91d03c4a6f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:32.248589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82147578-b281-464d-a701-ae91d03c4a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:32.248947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82147578-b281-464d-a701-ae91d03c4a6f admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:32.250023 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82147578-b281-464d-a701-ae91d03c4a6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:30:32.250430 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82147578-b281-464d-a701-ae91d03c4a6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:32.252954 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82147578-b281-464d-a701-ae91d03c4a6f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:32.265964 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1092/2204] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:30:32.271312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab415b-0c5e-4bea-bf04-3c47ad32b5fb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:32.281906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab415b-0c5e-4bea-bf04-3c47ad32b5fb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:32.288136 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510c8a8-0ea1-462e-a105-2d3f7b2afc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:32.288854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510c8a8-0ea1-462e-a105-2d3f7b2afc00 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:32.289146 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510c8a8-0ea1-462e-a105-2d3f7b2afc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:32.289428 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510c8a8-0ea1-462e-a105-2d3f7b2afc00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:32.290150 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a510c8a8-0ea1-462e-a105-2d3f7b2afc00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:32.292094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a510c8a8-0ea1-462e-a105-2d3f7b2afc00 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:32.326942 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1113/2205] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:30:32 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:32.331342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82147578-b281-464d-a701-ae91d03c4a6f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:32.342850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82147578-b281-464d-a701-ae91d03c4a6f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:32.409616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b435c7-be45-4d23-ae16-41e6ba5368fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:32.410288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b435c7-be45-4d23-ae16-41e6ba5368fa admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:32.410576 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b435c7-be45-4d23-ae16-41e6ba5368fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:32.410852 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b435c7-be45-4d23-ae16-41e6ba5368fa admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:32.411541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b435c7-be45-4d23-ae16-41e6ba5368fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:32.413498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b435c7-be45-4d23-ae16-41e6ba5368fa admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:32.433996 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1093/2206] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:30:32 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:32.438201 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510c8a8-0ea1-462e-a105-2d3f7b2afc00 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:32.449341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510c8a8-0ea1-462e-a105-2d3f7b2afc00 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:32.454478 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea5907-0ed8-4388-93cc-6a9e65aabb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:32.455084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea5907-0ed8-4388-93cc-6a9e65aabb07 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:32.455512 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea5907-0ed8-4388-93cc-6a9e65aabb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:32.455805 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea5907-0ed8-4388-93cc-6a9e65aabb07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:32.456613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea5907-0ed8-4388-93cc-6a9e65aabb07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:32.458248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea5907-0ed8-4388-93cc-6a9e65aabb07 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:32.469510 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1114/2207] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:30:32.473597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b435c7-be45-4d23-ae16-41e6ba5368fa admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:32.484785 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b435c7-be45-4d23-ae16-41e6ba5368fa admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:32.490081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bf825-5716-4db1-b952-0f2271f536ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:32.490693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bf825-5716-4db1-b952-0f2271f536ca admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:32.490982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bf825-5716-4db1-b952-0f2271f536ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:32.491269 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bf825-5716-4db1-b952-0f2271f536ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:32.492001 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bf825-5716-4db1-b952-0f2271f536ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:32.493726 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bf825-5716-4db1-b952-0f2271f536ca admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:32.504660 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1094/2208] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:30:32.508855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea5907-0ed8-4388-93cc-6a9e65aabb07 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:32.519051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea5907-0ed8-4388-93cc-6a9e65aabb07 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:32.526684 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ef376-c95a-4bcb-8121-c429ac8310f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:32.527738 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ef376-c95a-4bcb-8121-c429ac8310f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:32.528015 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ef376-c95a-4bcb-8121-c429ac8310f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:32.528289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ef376-c95a-4bcb-8121-c429ac8310f2 admin admin] PATH_INFO: `/v3/projects/c224dec664c64421944af07cf7cec856/users/8aadb4b4096d4cc5a5d46125eb63247c/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:32.546627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ef376-c95a-4bcb-8121-c429ac8310f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c224dec664c64421944af07cf7cec856, user_id=8aadb4b4096d4cc5a5d46125eb63247c, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:32.549938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ef376-c95a-4bcb-8121-c429ac8310f2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:32.563191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1115/2209] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:32 2022] PUT /identity/v3/projects/c224dec664c64421944af07cf7cec856/users/8aadb4b4096d4cc5a5d46125eb63247c/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:32.573280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975c380-d3fa-4579-b81f-541a560291fa None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:32.573580 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975c380-d3fa-4579-b81f-541a560291fa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:32.574597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975c380-d3fa-4579-b81f-541a560291fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:32.598862 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:32.609493 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e975c380-d3fa-4579-b81f-541a560291fa None None] MFA Rules not processed for user `8aadb4b4096d4cc5a5d46125eb63247c`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:32.610679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e975c380-d3fa-4579-b81f-541a560291fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:32.710152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1095/2210] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:32.722944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ef376-c95a-4bcb-8121-c429ac8310f2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:32.724818 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-906ef376-c95a-4bcb-8121-c429ac8310f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:32.836908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ef376-c95a-4bcb-8121-c429ac8310f2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:32.847604 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd023ca0-4c1b-446e-bee3-630fc51baa6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:32.848660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd023ca0-4c1b-446e-bee3-630fc51baa6d service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:32.848660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd023ca0-4c1b-446e-bee3-630fc51baa6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:32.848855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd023ca0-4c1b-446e-bee3-630fc51baa6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:32.877750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd023ca0-4c1b-446e-bee3-630fc51baa6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:32.879271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd023ca0-4c1b-446e-bee3-630fc51baa6d service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:32.897455 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1116/2211] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:32.925246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975c380-d3fa-4579-b81f-541a560291fa None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:33.009483 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975c380-d3fa-4579-b81f-541a560291fa None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:33.014679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ea389-edaa-4993-9af8-6ce7dbcd08f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:33.016093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ea389-edaa-4993-9af8-6ce7dbcd08f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:33.016382 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ea389-edaa-4993-9af8-6ce7dbcd08f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:33.016776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ea389-edaa-4993-9af8-6ce7dbcd08f2 admin admin] PATH_INFO: `/v3/users/8aadb4b4096d4cc5a5d46125eb63247c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:33.018362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ea389-edaa-4993-9af8-6ce7dbcd08f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aadb4b4096d4cc5a5d46125eb63247c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:33.020368 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ea389-edaa-4993-9af8-6ce7dbcd08f2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:33.108564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1a6ea389-edaa-4993-9af8-6ce7dbcd08f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aadb4b4096d4cc5a5d46125eb63247c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:33.109850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1a6ea389-edaa-4993-9af8-6ce7dbcd08f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aadb4b4096d4cc5a5d46125eb63247c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:33.110699 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1a6ea389-edaa-4993-9af8-6ce7dbcd08f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aadb4b4096d4cc5a5d46125eb63247c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:33.120250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1a6ea389-edaa-4993-9af8-6ce7dbcd08f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aadb4b4096d4cc5a5d46125eb63247c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:33.138540 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1a6ea389-edaa-4993-9af8-6ce7dbcd08f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aadb4b4096d4cc5a5d46125eb63247c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:33.157354 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1096/2212] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:32 2022] DELETE /identity/v3/users/8aadb4b4096d4cc5a5d46125eb63247c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:33.162293 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637d39c-cdb4-423b-8c4e-4e25f4edcdc0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:33.162293 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637d39c-cdb4-423b-8c4e-4e25f4edcdc0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:33.162565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637d39c-cdb4-423b-8c4e-4e25f4edcdc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:33.181352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2637d39c-cdb4-423b-8c4e-4e25f4edcdc0 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:33.182869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2637d39c-cdb4-423b-8c4e-4e25f4edcdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:33.197528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:33.264442 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1117/2213] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:30:33.275749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ea389-edaa-4993-9af8-6ce7dbcd08f2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:33.277539 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1a6ea389-edaa-4993-9af8-6ce7dbcd08f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:33.301169 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:33.343445 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ea389-edaa-4993-9af8-6ce7dbcd08f2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:33.348978 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add9dde-1e4e-4734-a54d-c21ec50deabb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:33.349647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add9dde-1e4e-4734-a54d-c21ec50deabb service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:33.349938 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add9dde-1e4e-4734-a54d-c21ec50deabb service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:33.350224 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add9dde-1e4e-4734-a54d-c21ec50deabb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:33.364439 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add9dde-1e4e-4734-a54d-c21ec50deabb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:33.365921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add9dde-1e4e-4734-a54d-c21ec50deabb service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:33.375503 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1097/2214] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:33.649527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637d39c-cdb4-423b-8c4e-4e25f4edcdc0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:33.651239 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2637d39c-cdb4-423b-8c4e-4e25f4edcdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:33.665358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637d39c-cdb4-423b-8c4e-4e25f4edcdc0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:33.670283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac623878-ffdb-4988-863b-8a4c72f55028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:33.671175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac623878-ffdb-4988-863b-8a4c72f55028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:33.671460 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac623878-ffdb-4988-863b-8a4c72f55028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:33.671732 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac623878-ffdb-4988-863b-8a4c72f55028 admin admin] PATH_INFO: `/v3/projects/c224dec664c64421944af07cf7cec856` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:33.672958 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac623878-ffdb-4988-863b-8a4c72f55028 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c224dec664c64421944af07cf7cec856)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:33.674916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac623878-ffdb-4988-863b-8a4c72f55028 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:33.737084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ac623878-ffdb-4988-863b-8a4c72f55028 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c224dec664c64421944af07cf7cec856', 'request_id': 'req-ac623878-ffdb-4988-863b-8a4c72f55028'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:33.743183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ac623878-ffdb-4988-863b-8a4c72f55028 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c224dec664c64421944af07cf7cec856', 'request_id': 'req-ac623878-ffdb-4988-863b-8a4c72f55028'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:33.752006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ac623878-ffdb-4988-863b-8a4c72f55028 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c224dec664c64421944af07cf7cec856', 'request_id': 'req-ac623878-ffdb-4988-863b-8a4c72f55028'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:33.764764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ac623878-ffdb-4988-863b-8a4c72f55028 admin admin] The token cache is being invalidate because project c224dec664c64421944af07cf7cec856 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:33.765057 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ac623878-ffdb-4988-863b-8a4c72f55028 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:33.765672 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ac623878-ffdb-4988-863b-8a4c72f55028 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:33.768250 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1118/2215] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:33 2022] DELETE /identity/v3/projects/c224dec664c64421944af07cf7cec856 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:34.692481 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add9dde-1e4e-4734-a54d-c21ec50deabb service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:34.700089 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2add9dde-1e4e-4734-a54d-c21ec50deabb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:34.717933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add9dde-1e4e-4734-a54d-c21ec50deabb service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:34.810418 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d908cf6-386d-464e-8823-d4fc02c69cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:34.811977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d908cf6-386d-464e-8823-d4fc02c69cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:34.813819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d908cf6-386d-464e-8823-d4fc02c69cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:34.813819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d908cf6-386d-464e-8823-d4fc02c69cbf admin admin] PATH_INFO: `/v3/users/856e8bb8a5f84e739cc3bc2d09d59aa6` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:34.815117 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d908cf6-386d-464e-8823-d4fc02c69cbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=856e8bb8a5f84e739cc3bc2d09d59aa6)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:34.817112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d908cf6-386d-464e-8823-d4fc02c69cbf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:34.885581 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d908cf6-386d-464e-8823-d4fc02c69cbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '856e8bb8a5f84e739cc3bc2d09d59aa6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:34.886263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d908cf6-386d-464e-8823-d4fc02c69cbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '856e8bb8a5f84e739cc3bc2d09d59aa6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:34.886924 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d908cf6-386d-464e-8823-d4fc02c69cbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '856e8bb8a5f84e739cc3bc2d09d59aa6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:34.899891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d908cf6-386d-464e-8823-d4fc02c69cbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '856e8bb8a5f84e739cc3bc2d09d59aa6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:34.917297 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d908cf6-386d-464e-8823-d4fc02c69cbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '856e8bb8a5f84e739cc3bc2d09d59aa6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:34.937790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1098/2216] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:34 2022] DELETE /identity/v3/users/856e8bb8a5f84e739cc3bc2d09d59aa6 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:30:34.942567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60340d-34fd-4974-9209-d19a0f75c7ae None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:34.943023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60340d-34fd-4974-9209-d19a0f75c7ae None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:34.943116 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60340d-34fd-4974-9209-d19a0f75c7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:34.962448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8d60340d-34fd-4974-9209-d19a0f75c7ae None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:34.963650 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8d60340d-34fd-4974-9209-d19a0f75c7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:35.034996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1119/2217] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:35.046931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d908cf6-386d-464e-8823-d4fc02c69cbf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:35.048969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3d908cf6-386d-464e-8823-d4fc02c69cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:35.098152 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:35.115615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d908cf6-386d-464e-8823-d4fc02c69cbf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:35.119500 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a4e8d-3278-4cca-b92d-c306246079aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:35.120160 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a4e8d-3278-4cca-b92d-c306246079aa service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:35.120442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a4e8d-3278-4cca-b92d-c306246079aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:35.120770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a4e8d-3278-4cca-b92d-c306246079aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:35.131502 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a4e8d-3278-4cca-b92d-c306246079aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:35.132982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a4e8d-3278-4cca-b92d-c306246079aa service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:35.143095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1099/2218] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:35.452413 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60340d-34fd-4974-9209-d19a0f75c7ae None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:35.456208 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8d60340d-34fd-4974-9209-d19a0f75c7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:35.475944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60340d-34fd-4974-9209-d19a0f75c7ae None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:35.489431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79156a5-1cf7-4dc9-9731-b869ae4f0ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:35.490587 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79156a5-1cf7-4dc9-9731-b869ae4f0ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:35.490837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79156a5-1cf7-4dc9-9731-b869ae4f0ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:35.491112 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79156a5-1cf7-4dc9-9731-b869ae4f0ce0 admin admin] PATH_INFO: `/v3/projects/1f6b50b861214381ae9aece1329dc6d8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:35.493512 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79156a5-1cf7-4dc9-9731-b869ae4f0ce0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f6b50b861214381ae9aece1329dc6d8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:35.495448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79156a5-1cf7-4dc9-9731-b869ae4f0ce0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:35.555148 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:35.568945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a79156a5-1cf7-4dc9-9731-b869ae4f0ce0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f6b50b861214381ae9aece1329dc6d8', 'request_id': 'req-a79156a5-1cf7-4dc9-9731-b869ae4f0ce0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:35.575993 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a79156a5-1cf7-4dc9-9731-b869ae4f0ce0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f6b50b861214381ae9aece1329dc6d8', 'request_id': 'req-a79156a5-1cf7-4dc9-9731-b869ae4f0ce0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:35.585552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a79156a5-1cf7-4dc9-9731-b869ae4f0ce0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f6b50b861214381ae9aece1329dc6d8', 'request_id': 'req-a79156a5-1cf7-4dc9-9731-b869ae4f0ce0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:35.603950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a79156a5-1cf7-4dc9-9731-b869ae4f0ce0 admin admin] The token cache is being invalidate because project 1f6b50b861214381ae9aece1329dc6d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:35.604239 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a79156a5-1cf7-4dc9-9731-b869ae4f0ce0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:35.605055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a79156a5-1cf7-4dc9-9731-b869ae4f0ce0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:35.607075 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1120/2219] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:35 2022] DELETE /identity/v3/projects/1f6b50b861214381ae9aece1329dc6d8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:35.611014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a4e8d-3278-4cca-b92d-c306246079aa service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:35.623457 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a4e8d-3278-4cca-b92d-c306246079aa service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:35.711172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7ac37-79da-46b0-a577-184be928e03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:35.712221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7ac37-79da-46b0-a577-184be928e03e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:35.712548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7ac37-79da-46b0-a577-184be928e03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:35.712854 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7ac37-79da-46b0-a577-184be928e03e admin admin] PATH_INFO: `/v3/users/c88ac6fc28104157a462c1a7a8c3b5f0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:35.714115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7ac37-79da-46b0-a577-184be928e03e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c88ac6fc28104157a462c1a7a8c3b5f0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:35.716066 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7ac37-79da-46b0-a577-184be928e03e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:35.790103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0da7ac37-79da-46b0-a577-184be928e03e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c88ac6fc28104157a462c1a7a8c3b5f0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:35.790912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0da7ac37-79da-46b0-a577-184be928e03e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c88ac6fc28104157a462c1a7a8c3b5f0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:35.792543 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0da7ac37-79da-46b0-a577-184be928e03e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c88ac6fc28104157a462c1a7a8c3b5f0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:35.801522 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0da7ac37-79da-46b0-a577-184be928e03e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c88ac6fc28104157a462c1a7a8c3b5f0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:35.818420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0da7ac37-79da-46b0-a577-184be928e03e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c88ac6fc28104157a462c1a7a8c3b5f0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:35.841016 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1100/2220] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:35 2022] DELETE /identity/v3/users/c88ac6fc28104157a462c1a7a8c3b5f0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:36.171679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79156a5-1cf7-4dc9-9731-b869ae4f0ce0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:36.175615 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a79156a5-1cf7-4dc9-9731-b869ae4f0ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:36.193265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79156a5-1cf7-4dc9-9731-b869ae4f0ce0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:36.257228 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86887e-9af5-4aee-a8aa-f5aef6b02cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:36.258150 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86887e-9af5-4aee-a8aa-f5aef6b02cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:36.258564 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86887e-9af5-4aee-a8aa-f5aef6b02cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:36.259061 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86887e-9af5-4aee-a8aa-f5aef6b02cb4 admin admin] PATH_INFO: `/v3/projects/c664ad8c4d4949e38c439952013554aa` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:36.260388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86887e-9af5-4aee-a8aa-f5aef6b02cb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c664ad8c4d4949e38c439952013554aa)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:36.262627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86887e-9af5-4aee-a8aa-f5aef6b02cb4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:36.313711 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6c86887e-9af5-4aee-a8aa-f5aef6b02cb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c664ad8c4d4949e38c439952013554aa', 'request_id': 'req-6c86887e-9af5-4aee-a8aa-f5aef6b02cb4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:36.319478 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6c86887e-9af5-4aee-a8aa-f5aef6b02cb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c664ad8c4d4949e38c439952013554aa', 'request_id': 'req-6c86887e-9af5-4aee-a8aa-f5aef6b02cb4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:36.325984 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6c86887e-9af5-4aee-a8aa-f5aef6b02cb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c664ad8c4d4949e38c439952013554aa', 'request_id': 'req-6c86887e-9af5-4aee-a8aa-f5aef6b02cb4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:36.339719 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6c86887e-9af5-4aee-a8aa-f5aef6b02cb4 admin admin] The token cache is being invalidate because project c664ad8c4d4949e38c439952013554aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:36.340065 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6c86887e-9af5-4aee-a8aa-f5aef6b02cb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:36.340768 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6c86887e-9af5-4aee-a8aa-f5aef6b02cb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:36.346233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1121/2221] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:36 2022] DELETE /identity/v3/projects/c664ad8c4d4949e38c439952013554aa => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:36.348551 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e61c96-8064-4dcf-ab9e-e15c9265d563 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:36.348551 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e61c96-8064-4dcf-ab9e-e15c9265d563 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:36.349034 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e61c96-8064-4dcf-ab9e-e15c9265d563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:36.370831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-45e61c96-8064-4dcf-ab9e-e15c9265d563 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:36.372077 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-45e61c96-8064-4dcf-ab9e-e15c9265d563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:36.448244 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1101/2222] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:36.456533 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22daf6f6-3ae5-410b-8840-d3f7732a9df2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:36.456845 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22daf6f6-3ae5-410b-8840-d3f7732a9df2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:36.457138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22daf6f6-3ae5-410b-8840-d3f7732a9df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:36.484003 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-22daf6f6-3ae5-410b-8840-d3f7732a9df2 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:36.485947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-22daf6f6-3ae5-410b-8840-d3f7732a9df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:36.495990 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1122/2223] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:30:36.500819 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e61c96-8064-4dcf-ab9e-e15c9265d563 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:36.506282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:36.512829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e61c96-8064-4dcf-ab9e-e15c9265d563 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:36.522151 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c35910-a995-47df-8b80-bfdfdd684b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:36.522971 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c35910-a995-47df-8b80-bfdfdd684b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:36.523195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c35910-a995-47df-8b80-bfdfdd684b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:36.523477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c35910-a995-47df-8b80-bfdfdd684b64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:36.524433 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c35910-a995-47df-8b80-bfdfdd684b64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:30:36.524841 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c35910-a995-47df-8b80-bfdfdd684b64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:36.526450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c35910-a995-47df-8b80-bfdfdd684b64 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:36.542472 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1102/2224] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:30:36.547300 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22daf6f6-3ae5-410b-8840-d3f7732a9df2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:36.559259 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22daf6f6-3ae5-410b-8840-d3f7732a9df2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:36.565398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602b335-8146-499e-b977-97eeb5cf2f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:36.566011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602b335-8146-499e-b977-97eeb5cf2f0b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:36.566325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602b335-8146-499e-b977-97eeb5cf2f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:36.566573 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602b335-8146-499e-b977-97eeb5cf2f0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:36.567284 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602b335-8146-499e-b977-97eeb5cf2f0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:36.569410 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602b335-8146-499e-b977-97eeb5cf2f0b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:36.600272 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1123/2225] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:30:36 2022] POST /identity/v3/projects => generated 393 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:36.604978 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c35910-a995-47df-8b80-bfdfdd684b64 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:36.617994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c35910-a995-47df-8b80-bfdfdd684b64 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:36.679937 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbb62e-89f2-42ac-899b-570f3548f529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:36.681015 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbb62e-89f2-42ac-899b-570f3548f529 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:36.681319 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbb62e-89f2-42ac-899b-570f3548f529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:36.681600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbb62e-89f2-42ac-899b-570f3548f529 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:36.682312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbb62e-89f2-42ac-899b-570f3548f529 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:36.684459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbb62e-89f2-42ac-899b-570f3548f529 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:36.708669 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1103/2226] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:30:36 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:30:36.712388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602b335-8146-499e-b977-97eeb5cf2f0b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:36.723321 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602b335-8146-499e-b977-97eeb5cf2f0b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:36.728774 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514af2f-9311-4623-914d-f44556c427a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:36.729521 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514af2f-9311-4623-914d-f44556c427a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:36.729811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514af2f-9311-4623-914d-f44556c427a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:36.730090 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514af2f-9311-4623-914d-f44556c427a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:36.730831 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514af2f-9311-4623-914d-f44556c427a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:36.732484 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514af2f-9311-4623-914d-f44556c427a0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:36.744200 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1124/2227] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:36.748544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbb62e-89f2-42ac-899b-570f3548f529 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:36.759903 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbb62e-89f2-42ac-899b-570f3548f529 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:36.764923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcaf63d-c8b3-4a98-9f9f-3236f2fdd4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:36.765589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcaf63d-c8b3-4a98-9f9f-3236f2fdd4de admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:36.765873 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcaf63d-c8b3-4a98-9f9f-3236f2fdd4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:36.766150 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcaf63d-c8b3-4a98-9f9f-3236f2fdd4de admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:36.766934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcaf63d-c8b3-4a98-9f9f-3236f2fdd4de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:36.768580 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcaf63d-c8b3-4a98-9f9f-3236f2fdd4de admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:36.779347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1104/2228] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:36.784379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514af2f-9311-4623-914d-f44556c427a0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:36.795288 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514af2f-9311-4623-914d-f44556c427a0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:36.801349 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01be8ed-0849-4d6c-87e1-5a96f524fc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:36.802269 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01be8ed-0849-4d6c-87e1-5a96f524fc2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:36.802555 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01be8ed-0849-4d6c-87e1-5a96f524fc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:36.802833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01be8ed-0849-4d6c-87e1-5a96f524fc2e admin admin] PATH_INFO: `/v3/projects/934da64f489b4da9bdbe8348d10178cd/users/40001ccffc684c3c81a1f3b6b8bb62f9/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:36.815542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:36.821334 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01be8ed-0849-4d6c-87e1-5a96f524fc2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=934da64f489b4da9bdbe8348d10178cd, user_id=40001ccffc684c3c81a1f3b6b8bb62f9, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:36.824674 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01be8ed-0849-4d6c-87e1-5a96f524fc2e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:36.839894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1125/2229] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:36 2022] PUT /identity/v3/projects/934da64f489b4da9bdbe8348d10178cd/users/40001ccffc684c3c81a1f3b6b8bb62f9/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:36.850214 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54534567-71d7-46dd-9013-674a0af19f90 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:36.850581 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54534567-71d7-46dd-9013-674a0af19f90 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:36.850930 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54534567-71d7-46dd-9013-674a0af19f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:36.885103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-54534567-71d7-46dd-9013-674a0af19f90 None None] MFA Rules not processed for user `40001ccffc684c3c81a1f3b6b8bb62f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:36.886403 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-54534567-71d7-46dd-9013-674a0af19f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:36.988313 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1105/2230] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:36.989663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01be8ed-0849-4d6c-87e1-5a96f524fc2e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:37.086004 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01be8ed-0849-4d6c-87e1-5a96f524fc2e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:37.102287 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef9fb9-70b7-4c69-93fc-8c624b96dfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:37.103295 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef9fb9-70b7-4c69-93fc-8c624b96dfe0 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:37.104034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef9fb9-70b7-4c69-93fc-8c624b96dfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:37.104645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef9fb9-70b7-4c69-93fc-8c624b96dfe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:37.105544 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef9fb9-70b7-4c69-93fc-8c624b96dfe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:37.107804 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef9fb9-70b7-4c69-93fc-8c624b96dfe0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:37.141638 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1126/2231] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:30:36 2022] POST /identity/v3/projects => generated 391 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:37.142833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54534567-71d7-46dd-9013-674a0af19f90 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:37.171227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54534567-71d7-46dd-9013-674a0af19f90 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:37.241140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:37.249940 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7faffa0-a0be-4db5-a95f-026574e1b7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:37.250935 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7faffa0-a0be-4db5-a95f-026574e1b7fe admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:37.251281 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7faffa0-a0be-4db5-a95f-026574e1b7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:37.251701 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7faffa0-a0be-4db5-a95f-026574e1b7fe admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:37.252493 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7faffa0-a0be-4db5-a95f-026574e1b7fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:37.254601 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7faffa0-a0be-4db5-a95f-026574e1b7fe admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:37.276334 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1106/2232] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:30:37 2022] POST /identity/v3/users => generated 302 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:37.280735 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef9fb9-70b7-4c69-93fc-8c624b96dfe0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:37.293090 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef9fb9-70b7-4c69-93fc-8c624b96dfe0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:37.299059 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1ff34-d0c5-4759-beb0-bd45632d4118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:37.300014 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1ff34-d0c5-4759-beb0-bd45632d4118 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:37.300613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1ff34-d0c5-4759-beb0-bd45632d4118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:37.300894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1ff34-d0c5-4759-beb0-bd45632d4118 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:37.301747 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1ff34-d0c5-4759-beb0-bd45632d4118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:37.303472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1ff34-d0c5-4759-beb0-bd45632d4118 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:37.319515 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1127/2233] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:30:37.320747 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7faffa0-a0be-4db5-a95f-026574e1b7fe admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:37.332070 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7faffa0-a0be-4db5-a95f-026574e1b7fe admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:37.337652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df529565-f24d-4ed9-b4bb-63305f1dd8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:37.338644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df529565-f24d-4ed9-b4bb-63305f1dd8d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:37.338818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df529565-f24d-4ed9-b4bb-63305f1dd8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:37.339100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df529565-f24d-4ed9-b4bb-63305f1dd8d2 admin admin] PATH_INFO: `/v3/projects/b4f0162750be423d8daf2dd94355e8df/users/f38f42a7570f4bf98000041aa7b6e326/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:37.358112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df529565-f24d-4ed9-b4bb-63305f1dd8d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4f0162750be423d8daf2dd94355e8df, user_id=f38f42a7570f4bf98000041aa7b6e326, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:37.362009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df529565-f24d-4ed9-b4bb-63305f1dd8d2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:37.381392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1107/2234] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:37 2022] PUT /identity/v3/projects/b4f0162750be423d8daf2dd94355e8df/users/f38f42a7570f4bf98000041aa7b6e326/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:30:37.385849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1ff34-d0c5-4759-beb0-bd45632d4118 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:37.398076 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1ff34-d0c5-4759-beb0-bd45632d4118 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:37.476532 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6adab5-f71c-4a17-a189-0412782fcad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:37.477218 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6adab5-f71c-4a17-a189-0412782fcad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:37.477506 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6adab5-f71c-4a17-a189-0412782fcad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:37.477850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6adab5-f71c-4a17-a189-0412782fcad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:37.478808 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6adab5-f71c-4a17-a189-0412782fcad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:37.480437 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6adab5-f71c-4a17-a189-0412782fcad3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:37.491334 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1128/2235] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:30:37.496210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df529565-f24d-4ed9-b4bb-63305f1dd8d2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:37.507154 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df529565-f24d-4ed9-b4bb-63305f1dd8d2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:37.513971 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf81a58-d9e0-49fc-8ae4-c19b5c4dbafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:37.514888 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf81a58-d9e0-49fc-8ae4-c19b5c4dbafb admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:37.515168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf81a58-d9e0-49fc-8ae4-c19b5c4dbafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:37.515438 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf81a58-d9e0-49fc-8ae4-c19b5c4dbafb admin admin] PATH_INFO: `/v3/projects/b4f0162750be423d8daf2dd94355e8df/users/f38f42a7570f4bf98000041aa7b6e326/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:37.518830 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf81a58-d9e0-49fc-8ae4-c19b5c4dbafb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4f0162750be423d8daf2dd94355e8df, user_id=f38f42a7570f4bf98000041aa7b6e326, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:37.521392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf81a58-d9e0-49fc-8ae4-c19b5c4dbafb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:37.534958 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1108/2236] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:37 2022] PUT /identity/v3/projects/b4f0162750be423d8daf2dd94355e8df/users/f38f42a7570f4bf98000041aa7b6e326/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:37.547229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799848c-0556-4784-9fec-beed10069c58 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:37.547531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799848c-0556-4784-9fec-beed10069c58 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:37.547826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799848c-0556-4784-9fec-beed10069c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:37.579185 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f799848c-0556-4784-9fec-beed10069c58 None None] MFA Rules not processed for user `f38f42a7570f4bf98000041aa7b6e326`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:37.581045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f799848c-0556-4784-9fec-beed10069c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:37.635055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:37.662469 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1129/2237] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:37 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:37.673870 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf81a58-d9e0-49fc-8ae4-c19b5c4dbafb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:37.675953 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bbf81a58-d9e0-49fc-8ae4-c19b5c4dbafb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:37.747259 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf81a58-d9e0-49fc-8ae4-c19b5c4dbafb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:37.752283 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80485dd1-cef3-4859-b0b7-11d5cc55c9bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:37.753012 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80485dd1-cef3-4859-b0b7-11d5cc55c9bc service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:37.753012 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80485dd1-cef3-4859-b0b7-11d5cc55c9bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:37.753812 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80485dd1-cef3-4859-b0b7-11d5cc55c9bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:37.765508 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80485dd1-cef3-4859-b0b7-11d5cc55c9bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:37.767044 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80485dd1-cef3-4859-b0b7-11d5cc55c9bc service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:37.784576 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1109/2238] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:37 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:37.846253 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799848c-0556-4784-9fec-beed10069c58 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:37.848718 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f799848c-0556-4784-9fec-beed10069c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:37.864696 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799848c-0556-4784-9fec-beed10069c58 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:37.868851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa852b-dade-4e60-872f-23c4f5e0401b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:37.869503 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa852b-dade-4e60-872f-23c4f5e0401b service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:37.869785 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa852b-dade-4e60-872f-23c4f5e0401b service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:37.870056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa852b-dade-4e60-872f-23c4f5e0401b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:37.951325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa852b-dade-4e60-872f-23c4f5e0401b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:37.952921 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa852b-dade-4e60-872f-23c4f5e0401b service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:37.963983 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1130/2239] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:41.519344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80485dd1-cef3-4859-b0b7-11d5cc55c9bc service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:41.622263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80485dd1-cef3-4859-b0b7-11d5cc55c9bc service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:41.631816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82935e55-3c30-4804-a49d-101179614ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:41.632806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82935e55-3c30-4804-a49d-101179614ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:41.633104 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82935e55-3c30-4804-a49d-101179614ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:41.633328 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82935e55-3c30-4804-a49d-101179614ac3 admin admin] PATH_INFO: `/v3/users/40001ccffc684c3c81a1f3b6b8bb62f9` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:41.634697 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82935e55-3c30-4804-a49d-101179614ac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40001ccffc684c3c81a1f3b6b8bb62f9)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:41.636682 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82935e55-3c30-4804-a49d-101179614ac3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:41.740322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-82935e55-3c30-4804-a49d-101179614ac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40001ccffc684c3c81a1f3b6b8bb62f9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:41.742380 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-82935e55-3c30-4804-a49d-101179614ac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40001ccffc684c3c81a1f3b6b8bb62f9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:41.744272 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-82935e55-3c30-4804-a49d-101179614ac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40001ccffc684c3c81a1f3b6b8bb62f9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:41.761582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:41.765073 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-82935e55-3c30-4804-a49d-101179614ac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40001ccffc684c3c81a1f3b6b8bb62f9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:41.809061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-82935e55-3c30-4804-a49d-101179614ac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40001ccffc684c3c81a1f3b6b8bb62f9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:41.852702 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1110/2240] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:41 2022] DELETE /identity/v3/users/40001ccffc684c3c81a1f3b6b8bb62f9 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:41.857607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825071c5-cd15-4162-b9c9-b6cc39862213 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:41.858460 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825071c5-cd15-4162-b9c9-b6cc39862213 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:41.866415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825071c5-cd15-4162-b9c9-b6cc39862213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:41.910675 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-825071c5-cd15-4162-b9c9-b6cc39862213 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:41.912344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-825071c5-cd15-4162-b9c9-b6cc39862213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:42.051875 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1131/2241] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:42.064924 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82935e55-3c30-4804-a49d-101179614ac3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:42.066363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-82935e55-3c30-4804-a49d-101179614ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:42.170555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82935e55-3c30-4804-a49d-101179614ac3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:42.178068 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45052cb-dd53-4ff6-8263-108314b98a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:42.178945 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45052cb-dd53-4ff6-8263-108314b98a42 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:42.179411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45052cb-dd53-4ff6-8263-108314b98a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:42.179878 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45052cb-dd53-4ff6-8263-108314b98a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:42.195250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45052cb-dd53-4ff6-8263-108314b98a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:42.197169 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45052cb-dd53-4ff6-8263-108314b98a42 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:42.210423 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1111/2242] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:42.616012 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825071c5-cd15-4162-b9c9-b6cc39862213 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:42.617494 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-825071c5-cd15-4162-b9c9-b6cc39862213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:42.633545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825071c5-cd15-4162-b9c9-b6cc39862213 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:42.651716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08842ae4-796c-4eba-88e9-50b96c0f939d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:42.652651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08842ae4-796c-4eba-88e9-50b96c0f939d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:42.653038 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08842ae4-796c-4eba-88e9-50b96c0f939d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:42.653200 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08842ae4-796c-4eba-88e9-50b96c0f939d admin admin] PATH_INFO: `/v3/projects/934da64f489b4da9bdbe8348d10178cd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:42.654957 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08842ae4-796c-4eba-88e9-50b96c0f939d admin admin] RBAC: Authorizing `identity:delete_project(project_id=934da64f489b4da9bdbe8348d10178cd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:42.660728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08842ae4-796c-4eba-88e9-50b96c0f939d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:42.716487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08842ae4-796c-4eba-88e9-50b96c0f939d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '934da64f489b4da9bdbe8348d10178cd', 'request_id': 'req-08842ae4-796c-4eba-88e9-50b96c0f939d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:42.722436 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08842ae4-796c-4eba-88e9-50b96c0f939d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '934da64f489b4da9bdbe8348d10178cd', 'request_id': 'req-08842ae4-796c-4eba-88e9-50b96c0f939d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:42.729967 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08842ae4-796c-4eba-88e9-50b96c0f939d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '934da64f489b4da9bdbe8348d10178cd', 'request_id': 'req-08842ae4-796c-4eba-88e9-50b96c0f939d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:42.745848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08842ae4-796c-4eba-88e9-50b96c0f939d admin admin] The token cache is being invalidate because project 934da64f489b4da9bdbe8348d10178cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:42.746267 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08842ae4-796c-4eba-88e9-50b96c0f939d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:42.746879 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08842ae4-796c-4eba-88e9-50b96c0f939d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:42.748708 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1132/2243] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:42 2022] DELETE /identity/v3/projects/934da64f489b4da9bdbe8348d10178cd => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:42.753202 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45052cb-dd53-4ff6-8263-108314b98a42 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:42.768150 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45052cb-dd53-4ff6-8263-108314b98a42 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:42.845030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf1f48-4a2f-46e3-a8ab-b7307612678e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:42.846046 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf1f48-4a2f-46e3-a8ab-b7307612678e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:42.846315 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf1f48-4a2f-46e3-a8ab-b7307612678e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:42.846604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf1f48-4a2f-46e3-a8ab-b7307612678e admin admin] PATH_INFO: `/v3/users/f38f42a7570f4bf98000041aa7b6e326` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:42.847913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf1f48-4a2f-46e3-a8ab-b7307612678e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f38f42a7570f4bf98000041aa7b6e326)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:42.849850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf1f48-4a2f-46e3-a8ab-b7307612678e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:42.927828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bdaf1f48-4a2f-46e3-a8ab-b7307612678e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f38f42a7570f4bf98000041aa7b6e326'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:42.928700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bdaf1f48-4a2f-46e3-a8ab-b7307612678e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f38f42a7570f4bf98000041aa7b6e326'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:42.930367 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bdaf1f48-4a2f-46e3-a8ab-b7307612678e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f38f42a7570f4bf98000041aa7b6e326'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:42.939537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bdaf1f48-4a2f-46e3-a8ab-b7307612678e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f38f42a7570f4bf98000041aa7b6e326'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:42.954133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-bdaf1f48-4a2f-46e3-a8ab-b7307612678e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f38f42a7570f4bf98000041aa7b6e326'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:42.959722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:42.973163 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1112/2244] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:42 2022] DELETE /identity/v3/users/f38f42a7570f4bf98000041aa7b6e326 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:43.324725 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08842ae4-796c-4eba-88e9-50b96c0f939d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:43.326945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-08842ae4-796c-4eba-88e9-50b96c0f939d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:43.354570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08842ae4-796c-4eba-88e9-50b96c0f939d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:43.368081 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:43.456591 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc9875-c23e-4a4a-a8a7-1768b5d7d954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:43.457471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc9875-c23e-4a4a-a8a7-1768b5d7d954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:43.457780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc9875-c23e-4a4a-a8a7-1768b5d7d954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:43.458064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc9875-c23e-4a4a-a8a7-1768b5d7d954 admin admin] PATH_INFO: `/v3/projects/b4f0162750be423d8daf2dd94355e8df` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:43.459256 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc9875-c23e-4a4a-a8a7-1768b5d7d954 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4f0162750be423d8daf2dd94355e8df)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:43.461289 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc9875-c23e-4a4a-a8a7-1768b5d7d954 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:43.512210 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-09cc9875-c23e-4a4a-a8a7-1768b5d7d954 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4f0162750be423d8daf2dd94355e8df', 'request_id': 'req-09cc9875-c23e-4a4a-a8a7-1768b5d7d954'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:43.524466 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-09cc9875-c23e-4a4a-a8a7-1768b5d7d954 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4f0162750be423d8daf2dd94355e8df', 'request_id': 'req-09cc9875-c23e-4a4a-a8a7-1768b5d7d954'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:43.531263 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-09cc9875-c23e-4a4a-a8a7-1768b5d7d954 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4f0162750be423d8daf2dd94355e8df', 'request_id': 'req-09cc9875-c23e-4a4a-a8a7-1768b5d7d954'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:43.544554 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-09cc9875-c23e-4a4a-a8a7-1768b5d7d954 admin admin] The token cache is being invalidate because project b4f0162750be423d8daf2dd94355e8df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:43.544850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-09cc9875-c23e-4a4a-a8a7-1768b5d7d954 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:43.545477 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-09cc9875-c23e-4a4a-a8a7-1768b5d7d954 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:43.550836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1133/2245] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:43 2022] DELETE /identity/v3/projects/b4f0162750be423d8daf2dd94355e8df => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:43.552653 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111d2d3-9e6b-4339-9681-46a7085bc7b0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:43.553112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111d2d3-9e6b-4339-9681-46a7085bc7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:43.553470 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111d2d3-9e6b-4339-9681-46a7085bc7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:43.572411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1111d2d3-9e6b-4339-9681-46a7085bc7b0 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:43.573663 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1111d2d3-9e6b-4339-9681-46a7085bc7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:43.633280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1113/2246] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:43.641529 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e25c4-8b95-4b4c-857c-fdb71a1ce994 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:43.641862 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e25c4-8b95-4b4c-857c-fdb71a1ce994 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:43.642145 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e25c4-8b95-4b4c-857c-fdb71a1ce994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:43.660749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f02e25c4-8b95-4b4c-857c-fdb71a1ce994 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:43.662044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f02e25c4-8b95-4b4c-857c-fdb71a1ce994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:43.670127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1134/2247] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:43.674832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111d2d3-9e6b-4339-9681-46a7085bc7b0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:43.684572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111d2d3-9e6b-4339-9681-46a7085bc7b0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:43.689715 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7565c01-2013-4d03-95ea-aa79e8bb2524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:43.690373 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7565c01-2013-4d03-95ea-aa79e8bb2524 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:43.690657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7565c01-2013-4d03-95ea-aa79e8bb2524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:43.690931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7565c01-2013-4d03-95ea-aa79e8bb2524 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:43.691659 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7565c01-2013-4d03-95ea-aa79e8bb2524 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:30:43.691970 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7565c01-2013-4d03-95ea-aa79e8bb2524 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:43.693693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7565c01-2013-4d03-95ea-aa79e8bb2524 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:43.705280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1114/2248] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:30:43.709677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e25c4-8b95-4b4c-857c-fdb71a1ce994 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:43.718764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e25c4-8b95-4b4c-857c-fdb71a1ce994 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:43.723427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071cc77e-9395-42ff-a831-85e3cc698ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:43.724103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071cc77e-9395-42ff-a831-85e3cc698ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:43.724389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071cc77e-9395-42ff-a831-85e3cc698ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:43.724693 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071cc77e-9395-42ff-a831-85e3cc698ba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:43.725393 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071cc77e-9395-42ff-a831-85e3cc698ba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:43.727307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071cc77e-9395-42ff-a831-85e3cc698ba2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:43.754021 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1135/2249] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:30:43 2022] POST /identity/v3/projects => generated 399 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:43.757935 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7565c01-2013-4d03-95ea-aa79e8bb2524 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:43.767825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7565c01-2013-4d03-95ea-aa79e8bb2524 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:43.821432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466586e-6d0a-4ca7-af26-6b984b2d4f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:43.822085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466586e-6d0a-4ca7-af26-6b984b2d4f30 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:43.822370 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466586e-6d0a-4ca7-af26-6b984b2d4f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:43.822646 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466586e-6d0a-4ca7-af26-6b984b2d4f30 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:43.823335 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466586e-6d0a-4ca7-af26-6b984b2d4f30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:43.825291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466586e-6d0a-4ca7-af26-6b984b2d4f30 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:43.852539 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1115/2250] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:30:43 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:30:43.856930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071cc77e-9395-42ff-a831-85e3cc698ba2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:43.869265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071cc77e-9395-42ff-a831-85e3cc698ba2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:43.873890 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dedfe-5955-4d7f-8d63-ca9cf481b939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:43.874552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dedfe-5955-4d7f-8d63-ca9cf481b939 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:43.874834 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dedfe-5955-4d7f-8d63-ca9cf481b939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:43.875109 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dedfe-5955-4d7f-8d63-ca9cf481b939 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:43.875862 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3dedfe-5955-4d7f-8d63-ca9cf481b939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:43.877530 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3dedfe-5955-4d7f-8d63-ca9cf481b939 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:43.887991 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1136/2251] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:43.892180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466586e-6d0a-4ca7-af26-6b984b2d4f30 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:43.902172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466586e-6d0a-4ca7-af26-6b984b2d4f30 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:43.906750 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d82fe-7673-448c-aa9a-04b3effdebd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:43.907404 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d82fe-7673-448c-aa9a-04b3effdebd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:43.908203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d82fe-7673-448c-aa9a-04b3effdebd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:43.908485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d82fe-7673-448c-aa9a-04b3effdebd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:43.909443 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d82fe-7673-448c-aa9a-04b3effdebd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:43.910838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d82fe-7673-448c-aa9a-04b3effdebd0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:43.921211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1116/2252] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:43.926030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dedfe-5955-4d7f-8d63-ca9cf481b939 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:43.935998 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dedfe-5955-4d7f-8d63-ca9cf481b939 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:43.940625 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8ac09-f020-44af-a9b9-219f3df5f822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:43.941505 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8ac09-f020-44af-a9b9-219f3df5f822 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:43.941791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8ac09-f020-44af-a9b9-219f3df5f822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:43.942067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8ac09-f020-44af-a9b9-219f3df5f822 admin admin] PATH_INFO: `/v3/projects/6468be1f96754b3190db1c1e08aa7ea3/users/89fe4cf34d5545ce9a20a1accc08cea5/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:43.956226 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8ac09-f020-44af-a9b9-219f3df5f822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6468be1f96754b3190db1c1e08aa7ea3, user_id=89fe4cf34d5545ce9a20a1accc08cea5, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:43.958916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8ac09-f020-44af-a9b9-219f3df5f822 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:43.971737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1137/2253] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:43 2022] PUT /identity/v3/projects/6468be1f96754b3190db1c1e08aa7ea3/users/89fe4cf34d5545ce9a20a1accc08cea5/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:43.983484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd6b6d-2b76-4fb9-a90c-458b5b84ae3c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:43.983782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd6b6d-2b76-4fb9-a90c-458b5b84ae3c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:43.984061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd6b6d-2b76-4fb9-a90c-458b5b84ae3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:44.011600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-21fd6b6d-2b76-4fb9-a90c-458b5b84ae3c None None] MFA Rules not processed for user `89fe4cf34d5545ce9a20a1accc08cea5`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:44.012830 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-21fd6b6d-2b76-4fb9-a90c-458b5b84ae3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:44.058582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:44.077658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1117/2254] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:30:44.082239 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8ac09-f020-44af-a9b9-219f3df5f822 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:44.117220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:44.146699 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8ac09-f020-44af-a9b9-219f3df5f822 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:44.151378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c8d2c-9689-43c9-a3f5-68c124fa6b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:44.152038 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c8d2c-9689-43c9-a3f5-68c124fa6b3d admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:44.152322 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c8d2c-9689-43c9-a3f5-68c124fa6b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:44.152628 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c8d2c-9689-43c9-a3f5-68c124fa6b3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:44.153324 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c8d2c-9689-43c9-a3f5-68c124fa6b3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:44.155241 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c8d2c-9689-43c9-a3f5-68c124fa6b3d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:44.180774 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1138/2255] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:30:44 2022] POST /identity/v3/projects => generated 399 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:44.184938 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd6b6d-2b76-4fb9-a90c-458b5b84ae3c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:44.195606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd6b6d-2b76-4fb9-a90c-458b5b84ae3c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:44.250219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37883a5c-395a-421b-8169-89e5de8a88e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:44.250961 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37883a5c-395a-421b-8169-89e5de8a88e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:44.251387 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37883a5c-395a-421b-8169-89e5de8a88e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:44.251669 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37883a5c-395a-421b-8169-89e5de8a88e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:44.252391 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37883a5c-395a-421b-8169-89e5de8a88e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:44.254334 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37883a5c-395a-421b-8169-89e5de8a88e7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:44.273976 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1118/2256] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:30:44 2022] POST /identity/v3/users => generated 306 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:30:44.277954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c8d2c-9689-43c9-a3f5-68c124fa6b3d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:44.288603 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c8d2c-9689-43c9-a3f5-68c124fa6b3d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:44.293350 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c296992-d2ba-4586-83ba-2068f99dfdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:44.294001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c296992-d2ba-4586-83ba-2068f99dfdc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:44.294283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c296992-d2ba-4586-83ba-2068f99dfdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:44.294559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c296992-d2ba-4586-83ba-2068f99dfdc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:44.295278 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c296992-d2ba-4586-83ba-2068f99dfdc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:44.296927 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c296992-d2ba-4586-83ba-2068f99dfdc1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:44.307189 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1139/2257] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:44.311141 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37883a5c-395a-421b-8169-89e5de8a88e7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:44.320914 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37883a5c-395a-421b-8169-89e5de8a88e7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:44.325454 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffcf4a-3a9e-4ac5-b8c8-526f7d81d007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:44.326325 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffcf4a-3a9e-4ac5-b8c8-526f7d81d007 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:44.326605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffcf4a-3a9e-4ac5-b8c8-526f7d81d007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:44.326880 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffcf4a-3a9e-4ac5-b8c8-526f7d81d007 admin admin] PATH_INFO: `/v3/projects/f7e58ddc77fa4b4c92abd086f3e4267c/users/1f2f7fc541b34be2969ab9c37431fa26/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:44.341509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ffcf4a-3a9e-4ac5-b8c8-526f7d81d007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e58ddc77fa4b4c92abd086f3e4267c, user_id=1f2f7fc541b34be2969ab9c37431fa26, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:44.344180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ffcf4a-3a9e-4ac5-b8c8-526f7d81d007 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:44.357567 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1119/2258] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:44 2022] PUT /identity/v3/projects/f7e58ddc77fa4b4c92abd086f3e4267c/users/1f2f7fc541b34be2969ab9c37431fa26/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:30:44.361417 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c296992-d2ba-4586-83ba-2068f99dfdc1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:44.371266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c296992-d2ba-4586-83ba-2068f99dfdc1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:44.428719 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4160ef1-6826-40ac-99e2-44b62e5c09b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:44.429464 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4160ef1-6826-40ac-99e2-44b62e5c09b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:44.430162 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4160ef1-6826-40ac-99e2-44b62e5c09b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:44.430162 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4160ef1-6826-40ac-99e2-44b62e5c09b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:44.431094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4160ef1-6826-40ac-99e2-44b62e5c09b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:44.432811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4160ef1-6826-40ac-99e2-44b62e5c09b1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:44.443362 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1140/2259] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:44.447598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffcf4a-3a9e-4ac5-b8c8-526f7d81d007 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:44.458543 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffcf4a-3a9e-4ac5-b8c8-526f7d81d007 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:44.463149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa36ea0-a94a-41bc-bb2c-cb6ac691fb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:44.464027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa36ea0-a94a-41bc-bb2c-cb6ac691fb8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:44.464313 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa36ea0-a94a-41bc-bb2c-cb6ac691fb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:44.464618 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa36ea0-a94a-41bc-bb2c-cb6ac691fb8a admin admin] PATH_INFO: `/v3/projects/f7e58ddc77fa4b4c92abd086f3e4267c/users/1f2f7fc541b34be2969ab9c37431fa26/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:44.466577 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa36ea0-a94a-41bc-bb2c-cb6ac691fb8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e58ddc77fa4b4c92abd086f3e4267c, user_id=1f2f7fc541b34be2969ab9c37431fa26, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:44.469249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa36ea0-a94a-41bc-bb2c-cb6ac691fb8a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:44.480381 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1120/2260] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:44 2022] PUT /identity/v3/projects/f7e58ddc77fa4b4c92abd086f3e4267c/users/1f2f7fc541b34be2969ab9c37431fa26/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:44.490423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4145ae-e288-472e-8afd-a5282bef6413 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:44.490714 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4145ae-e288-472e-8afd-a5282bef6413 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:44.491018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4145ae-e288-472e-8afd-a5282bef6413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:44.516724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7f4145ae-e288-472e-8afd-a5282bef6413 None None] MFA Rules not processed for user `1f2f7fc541b34be2969ab9c37431fa26`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:44.518480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7f4145ae-e288-472e-8afd-a5282bef6413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:44.589330 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1141/2261] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:44 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:44.600206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa36ea0-a94a-41bc-bb2c-cb6ac691fb8a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:44.602074 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6fa36ea0-a94a-41bc-bb2c-cb6ac691fb8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:44.661428 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa36ea0-a94a-41bc-bb2c-cb6ac691fb8a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:44.665157 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca3fe7-0806-47f4-8d04-ec3ee8882260 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:44.665815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca3fe7-0806-47f4-8d04-ec3ee8882260 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:44.666090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca3fe7-0806-47f4-8d04-ec3ee8882260 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:44.666874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca3fe7-0806-47f4-8d04-ec3ee8882260 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:44.675522 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca3fe7-0806-47f4-8d04-ec3ee8882260 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:44.677012 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca3fe7-0806-47f4-8d04-ec3ee8882260 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:44.686794 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1121/2262] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:44 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:44.766731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4145ae-e288-472e-8afd-a5282bef6413 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:44.776691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4145ae-e288-472e-8afd-a5282bef6413 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:44.780234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee2991-b9b8-4d18-9fd0-f704c3513486 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:44.780917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee2991-b9b8-4d18-9fd0-f704c3513486 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:44.781203 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee2991-b9b8-4d18-9fd0-f704c3513486 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:44.781504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee2991-b9b8-4d18-9fd0-f704c3513486 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:44.809786 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:44.841808 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee2991-b9b8-4d18-9fd0-f704c3513486 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:44.843182 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee2991-b9b8-4d18-9fd0-f704c3513486 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:44.852728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1142/2263] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:47.286406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc35dd-e66f-4a75-8444-e569276edc12 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:47.286845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc35dd-e66f-4a75-8444-e569276edc12 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:47.286977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc35dd-e66f-4a75-8444-e569276edc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:47.311364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-dcfc35dd-e66f-4a75-8444-e569276edc12 None None] MFA Rules not processed for user `a09029144c454b099bea86aaa5229ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:47.312639 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dcfc35dd-e66f-4a75-8444-e569276edc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:47.355317 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:47.371206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1122/2264] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 18 10:30:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:47.417565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee2991-b9b8-4d18-9fd0-f704c3513486 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:47.419001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a7ee2991-b9b8-4d18-9fd0-f704c3513486 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:47.494898 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee2991-b9b8-4d18-9fd0-f704c3513486 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:47.504823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d131e-a698-4a1e-bedc-f738819f10fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6168873f2846c8af73b903bba17eb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:47.505342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d131e-a698-4a1e-bedc-f738819f10fa service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:47.505736 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d131e-a698-4a1e-bedc-f738819f10fa service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:47.506085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d131e-a698-4a1e-bedc-f738819f10fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:47.515689 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d131e-a698-4a1e-bedc-f738819f10fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:47.518000 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d131e-a698-4a1e-bedc-f738819f10fa service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:47.527048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1143/2265] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 18 10:30:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:30:50.112967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc35dd-e66f-4a75-8444-e569276edc12 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:50.201010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc35dd-e66f-4a75-8444-e569276edc12 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:50.206012 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb885f-2145-4d48-8ce4-e3b7955a846c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:50.206881 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb885f-2145-4d48-8ce4-e3b7955a846c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:50.207342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb885f-2145-4d48-8ce4-e3b7955a846c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:50.207626 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb885f-2145-4d48-8ce4-e3b7955a846c admin admin] PATH_INFO: `/v3/users/89fe4cf34d5545ce9a20a1accc08cea5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:50.208886 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decb885f-2145-4d48-8ce4-e3b7955a846c admin admin] RBAC: Authorizing `identity:delete_user(user_id=89fe4cf34d5545ce9a20a1accc08cea5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:50.210938 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decb885f-2145-4d48-8ce4-e3b7955a846c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:50.333483 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-decb885f-2145-4d48-8ce4-e3b7955a846c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89fe4cf34d5545ce9a20a1accc08cea5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:50.334425 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-decb885f-2145-4d48-8ce4-e3b7955a846c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89fe4cf34d5545ce9a20a1accc08cea5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:50.335231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-decb885f-2145-4d48-8ce4-e3b7955a846c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89fe4cf34d5545ce9a20a1accc08cea5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:50.346302 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-decb885f-2145-4d48-8ce4-e3b7955a846c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89fe4cf34d5545ce9a20a1accc08cea5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:50.361311 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-decb885f-2145-4d48-8ce4-e3b7955a846c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89fe4cf34d5545ce9a20a1accc08cea5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:50.378874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1123/2266] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:50 2022] DELETE /identity/v3/users/89fe4cf34d5545ce9a20a1accc08cea5 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:50.384107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8c8f7-e56f-4aef-b976-cb89c8c3263a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:50.385116 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8c8f7-e56f-4aef-b976-cb89c8c3263a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:50.385745 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8c8f7-e56f-4aef-b976-cb89c8c3263a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:50.408262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-34f8c8f7-e56f-4aef-b976-cb89c8c3263a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:50.410586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-34f8c8f7-e56f-4aef-b976-cb89c8c3263a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:50.483001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1144/2267] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:50.493949 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb885f-2145-4d48-8ce4-e3b7955a846c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:50.495639 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-decb885f-2145-4d48-8ce4-e3b7955a846c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:50.572387 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb885f-2145-4d48-8ce4-e3b7955a846c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:50.576499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09d4cb-6a9c-4533-843f-5d855f0ae9c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:50.577279 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09d4cb-6a9c-4533-843f-5d855f0ae9c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:50.577642 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09d4cb-6a9c-4533-843f-5d855f0ae9c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:50.577995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09d4cb-6a9c-4533-843f-5d855f0ae9c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:50.588806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b09d4cb-6a9c-4533-843f-5d855f0ae9c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:50.590396 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b09d4cb-6a9c-4533-843f-5d855f0ae9c0 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:50.618222 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1124/2268] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:30:51.062928 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8c8f7-e56f-4aef-b976-cb89c8c3263a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:51.065176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-34f8c8f7-e56f-4aef-b976-cb89c8c3263a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:51.087544 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8c8f7-e56f-4aef-b976-cb89c8c3263a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:51.096993 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7c48a-dd6e-45bb-9570-374c547ef535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:51.098181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7c48a-dd6e-45bb-9570-374c547ef535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:51.098534 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7c48a-dd6e-45bb-9570-374c547ef535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:51.098939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7c48a-dd6e-45bb-9570-374c547ef535 admin admin] PATH_INFO: `/v3/projects/6468be1f96754b3190db1c1e08aa7ea3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:51.100420 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7c48a-dd6e-45bb-9570-374c547ef535 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6468be1f96754b3190db1c1e08aa7ea3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:51.102749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7c48a-dd6e-45bb-9570-374c547ef535 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:51.168449 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-57e7c48a-dd6e-45bb-9570-374c547ef535 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6468be1f96754b3190db1c1e08aa7ea3', 'request_id': 'req-57e7c48a-dd6e-45bb-9570-374c547ef535'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:51.170530 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:51.177946 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-57e7c48a-dd6e-45bb-9570-374c547ef535 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6468be1f96754b3190db1c1e08aa7ea3', 'request_id': 'req-57e7c48a-dd6e-45bb-9570-374c547ef535'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:51.190464 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-57e7c48a-dd6e-45bb-9570-374c547ef535 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6468be1f96754b3190db1c1e08aa7ea3', 'request_id': 'req-57e7c48a-dd6e-45bb-9570-374c547ef535'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:51.210295 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-57e7c48a-dd6e-45bb-9570-374c547ef535 admin admin] The token cache is being invalidate because project 6468be1f96754b3190db1c1e08aa7ea3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:51.210712 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-57e7c48a-dd6e-45bb-9570-374c547ef535 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:51.211676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-57e7c48a-dd6e-45bb-9570-374c547ef535 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:51.213947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1145/2269] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:51 2022] DELETE /identity/v3/projects/6468be1f96754b3190db1c1e08aa7ea3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:51.217726 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09d4cb-6a9c-4533-843f-5d855f0ae9c0 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:51.228273 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09d4cb-6a9c-4533-843f-5d855f0ae9c0 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:51.237888 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:51.307657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f440c67-bc82-4567-a2c1-6273a7d8484a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:51.308623 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f440c67-bc82-4567-a2c1-6273a7d8484a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:51.309002 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f440c67-bc82-4567-a2c1-6273a7d8484a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:51.309325 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f440c67-bc82-4567-a2c1-6273a7d8484a admin admin] PATH_INFO: `/v3/users/1f2f7fc541b34be2969ab9c37431fa26` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:51.310942 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f440c67-bc82-4567-a2c1-6273a7d8484a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f2f7fc541b34be2969ab9c37431fa26)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:51.313626 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f440c67-bc82-4567-a2c1-6273a7d8484a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:51.386902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8f440c67-bc82-4567-a2c1-6273a7d8484a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f2f7fc541b34be2969ab9c37431fa26'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:51.387877 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8f440c67-bc82-4567-a2c1-6273a7d8484a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f2f7fc541b34be2969ab9c37431fa26'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:51.388733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8f440c67-bc82-4567-a2c1-6273a7d8484a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f2f7fc541b34be2969ab9c37431fa26'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:51.403091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8f440c67-bc82-4567-a2c1-6273a7d8484a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f2f7fc541b34be2969ab9c37431fa26'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:51.420354 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8f440c67-bc82-4567-a2c1-6273a7d8484a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f2f7fc541b34be2969ab9c37431fa26'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:51.448000 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1125/2270] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:51 2022] DELETE /identity/v3/users/1f2f7fc541b34be2969ab9c37431fa26 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:51.961003 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7c48a-dd6e-45bb-9570-374c547ef535 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:51.963228 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-57e7c48a-dd6e-45bb-9570-374c547ef535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:51.994803 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7c48a-dd6e-45bb-9570-374c547ef535 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:52.117133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08132eda-8451-4a3c-8680-38f04e99417b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:52.118030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08132eda-8451-4a3c-8680-38f04e99417b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:52.118316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08132eda-8451-4a3c-8680-38f04e99417b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:52.118590 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08132eda-8451-4a3c-8680-38f04e99417b admin admin] PATH_INFO: `/v3/projects/f7e58ddc77fa4b4c92abd086f3e4267c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:52.119969 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08132eda-8451-4a3c-8680-38f04e99417b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7e58ddc77fa4b4c92abd086f3e4267c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:52.121831 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08132eda-8451-4a3c-8680-38f04e99417b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:52.192813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08132eda-8451-4a3c-8680-38f04e99417b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7e58ddc77fa4b4c92abd086f3e4267c', 'request_id': 'req-08132eda-8451-4a3c-8680-38f04e99417b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:52.200406 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08132eda-8451-4a3c-8680-38f04e99417b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7e58ddc77fa4b4c92abd086f3e4267c', 'request_id': 'req-08132eda-8451-4a3c-8680-38f04e99417b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:52.208182 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08132eda-8451-4a3c-8680-38f04e99417b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7e58ddc77fa4b4c92abd086f3e4267c', 'request_id': 'req-08132eda-8451-4a3c-8680-38f04e99417b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:52.229334 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08132eda-8451-4a3c-8680-38f04e99417b admin admin] The token cache is being invalidate because project f7e58ddc77fa4b4c92abd086f3e4267c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:52.230065 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08132eda-8451-4a3c-8680-38f04e99417b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:52.231582 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-08132eda-8451-4a3c-8680-38f04e99417b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:52.235242 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1146/2271] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:51 2022] DELETE /identity/v3/projects/f7e58ddc77fa4b4c92abd086f3e4267c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:52.242337 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965629-380e-4adb-8d00-599d61c293ae None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:52.242871 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965629-380e-4adb-8d00-599d61c293ae None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:52.243527 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965629-380e-4adb-8d00-599d61c293ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:52.280597 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ce965629-380e-4adb-8d00-599d61c293ae None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:52.283911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ce965629-380e-4adb-8d00-599d61c293ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:52.367534 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1126/2272] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:52.379685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f174df6-22b1-46a1-9785-7be5d38d1d27 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:52.380318 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f174df6-22b1-46a1-9785-7be5d38d1d27 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:52.381212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f174df6-22b1-46a1-9785-7be5d38d1d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:52.413873 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4f174df6-22b1-46a1-9785-7be5d38d1d27 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:52.416680 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4f174df6-22b1-46a1-9785-7be5d38d1d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:52.425650 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1147/2273] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:30:52.429704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce965629-380e-4adb-8d00-599d61c293ae None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:52.446977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce965629-380e-4adb-8d00-599d61c293ae None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:52.461294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fe5c0-cfc0-46e7-ae1c-6b244e762662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:52.462090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fe5c0-cfc0-46e7-ae1c-6b244e762662 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:52.462447 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fe5c0-cfc0-46e7-ae1c-6b244e762662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:52.462806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fe5c0-cfc0-46e7-ae1c-6b244e762662 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:52.463633 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fe5c0-cfc0-46e7-ae1c-6b244e762662 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:30:52.464021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fe5c0-cfc0-46e7-ae1c-6b244e762662 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:52.465811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fe5c0-cfc0-46e7-ae1c-6b244e762662 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:52.483378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1127/2274] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:30:52.488003 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f174df6-22b1-46a1-9785-7be5d38d1d27 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:52.503196 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f174df6-22b1-46a1-9785-7be5d38d1d27 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:52.508613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56717d-490c-44cf-b574-be4114bb1d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:52.509357 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56717d-490c-44cf-b574-be4114bb1d7f admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:52.509823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56717d-490c-44cf-b574-be4114bb1d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:52.510563 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56717d-490c-44cf-b574-be4114bb1d7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:52.511886 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb56717d-490c-44cf-b574-be4114bb1d7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:52.517663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb56717d-490c-44cf-b574-be4114bb1d7f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:52.564075 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1148/2275] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:30:52 2022] POST /identity/v3/projects => generated 375 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:52.571137 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fe5c0-cfc0-46e7-ae1c-6b244e762662 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:52.582643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fe5c0-cfc0-46e7-ae1c-6b244e762662 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:52.599039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:52.664138 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4bf3b-31b9-49e3-bc53-2a97905691d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:52.665016 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4bf3b-31b9-49e3-bc53-2a97905691d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:52.665375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4bf3b-31b9-49e3-bc53-2a97905691d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:52.667470 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4bf3b-31b9-49e3-bc53-2a97905691d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:52.668274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4bf3b-31b9-49e3-bc53-2a97905691d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:52.670301 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4bf3b-31b9-49e3-bc53-2a97905691d6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:52.694882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1128/2276] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:30:52 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:52.699112 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56717d-490c-44cf-b574-be4114bb1d7f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:52.714637 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56717d-490c-44cf-b574-be4114bb1d7f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:52.720202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ae2fe-7d00-4d55-82b1-d12f45685b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:52.721203 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ae2fe-7d00-4d55-82b1-d12f45685b15 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:52.721682 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ae2fe-7d00-4d55-82b1-d12f45685b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:52.722165 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ae2fe-7d00-4d55-82b1-d12f45685b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:52.723132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ae2fe-7d00-4d55-82b1-d12f45685b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:52.725159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ae2fe-7d00-4d55-82b1-d12f45685b15 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:52.738855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1149/2277] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:52.743440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4bf3b-31b9-49e3-bc53-2a97905691d6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:52.755503 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4bf3b-31b9-49e3-bc53-2a97905691d6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:52.761170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7887e-556d-44d0-9fa0-14e804c19905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:52.762013 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7887e-556d-44d0-9fa0-14e804c19905 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:52.762403 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7887e-556d-44d0-9fa0-14e804c19905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:52.762786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7887e-556d-44d0-9fa0-14e804c19905 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:52.763573 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7887e-556d-44d0-9fa0-14e804c19905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:52.765825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7887e-556d-44d0-9fa0-14e804c19905 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:52.786189 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1129/2278] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:52.787898 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ae2fe-7d00-4d55-82b1-d12f45685b15 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:52.803681 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ae2fe-7d00-4d55-82b1-d12f45685b15 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:52.810097 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c8665-c717-4d97-83f3-297c312db666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:52.810996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c8665-c717-4d97-83f3-297c312db666 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:52.811275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c8665-c717-4d97-83f3-297c312db666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:52.811546 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c8665-c717-4d97-83f3-297c312db666 admin admin] PATH_INFO: `/v3/projects/9044f5482f98459a93f1a186254196c0/users/50de117e09e04d12ab05aa5e96fe7b00/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:52.828780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c8665-c717-4d97-83f3-297c312db666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9044f5482f98459a93f1a186254196c0, user_id=50de117e09e04d12ab05aa5e96fe7b00, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:52.831460 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c8665-c717-4d97-83f3-297c312db666 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:52.858823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1150/2279] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:52 2022] PUT /identity/v3/projects/9044f5482f98459a93f1a186254196c0/users/50de117e09e04d12ab05aa5e96fe7b00/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:52.876048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e667c5-1ec3-4059-9ec4-2a7a5443c423 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:52.876730 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e667c5-1ec3-4059-9ec4-2a7a5443c423 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:52.877166 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e667c5-1ec3-4059-9ec4-2a7a5443c423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:52.908873 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-77e667c5-1ec3-4059-9ec4-2a7a5443c423 None None] MFA Rules not processed for user `50de117e09e04d12ab05aa5e96fe7b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:52.910361 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-77e667c5-1ec3-4059-9ec4-2a7a5443c423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:52.988650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1130/2280] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:52.999792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c8665-c717-4d97-83f3-297c312db666 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:53.001794 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-917c8665-c717-4d97-83f3-297c312db666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:53.014615 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:53.076604 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c8665-c717-4d97-83f3-297c312db666 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:53.081229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74bba1-f222-4a17-b0d8-406977110155 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:53.081964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74bba1-f222-4a17-b0d8-406977110155 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:53.082323 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74bba1-f222-4a17-b0d8-406977110155 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:53.082691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74bba1-f222-4a17-b0d8-406977110155 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:53.092130 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74bba1-f222-4a17-b0d8-406977110155 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:53.093797 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74bba1-f222-4a17-b0d8-406977110155 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:53.109957 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1151/2281] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:56.947712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e667c5-1ec3-4059-9ec4-2a7a5443c423 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:57.043940 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e667c5-1ec3-4059-9ec4-2a7a5443c423 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:57.050146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c4985-6896-43b2-b8e6-d0244d28ecf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:57.051309 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c4985-6896-43b2-b8e6-d0244d28ecf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:57.051873 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c4985-6896-43b2-b8e6-d0244d28ecf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:57.052548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c4985-6896-43b2-b8e6-d0244d28ecf0 admin admin] PATH_INFO: `/v3/users/50de117e09e04d12ab05aa5e96fe7b00` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:57.054157 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c4985-6896-43b2-b8e6-d0244d28ecf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50de117e09e04d12ab05aa5e96fe7b00)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:57.056354 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c4985-6896-43b2-b8e6-d0244d28ecf0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:57.113136 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:57.157198 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3e5c4985-6896-43b2-b8e6-d0244d28ecf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50de117e09e04d12ab05aa5e96fe7b00'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:57.160979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3e5c4985-6896-43b2-b8e6-d0244d28ecf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50de117e09e04d12ab05aa5e96fe7b00'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:57.161094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3e5c4985-6896-43b2-b8e6-d0244d28ecf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50de117e09e04d12ab05aa5e96fe7b00'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:57.173489 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3e5c4985-6896-43b2-b8e6-d0244d28ecf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50de117e09e04d12ab05aa5e96fe7b00'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:57.205959 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3e5c4985-6896-43b2-b8e6-d0244d28ecf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50de117e09e04d12ab05aa5e96fe7b00'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:57.236210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1131/2282] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:30:56 2022] DELETE /identity/v3/users/50de117e09e04d12ab05aa5e96fe7b00 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:57.240444 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79184ff-214a-4fe1-9915-39e32a92899c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:57.240444 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79184ff-214a-4fe1-9915-39e32a92899c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:57.241538 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79184ff-214a-4fe1-9915-39e32a92899c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:57.270997 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e79184ff-214a-4fe1-9915-39e32a92899c None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:57.272606 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e79184ff-214a-4fe1-9915-39e32a92899c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:57.389671 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1152/2283] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:57.400433 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c4985-6896-43b2-b8e6-d0244d28ecf0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:57.401910 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3e5c4985-6896-43b2-b8e6-d0244d28ecf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:57.530778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c4985-6896-43b2-b8e6-d0244d28ecf0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:57.535882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ce698-64a4-4aed-8c94-49d8e76cc687 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:57.536912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ce698-64a4-4aed-8c94-49d8e76cc687 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:57.537417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ce698-64a4-4aed-8c94-49d8e76cc687 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:57.537893 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ce698-64a4-4aed-8c94-49d8e76cc687 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:57.551495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ce698-64a4-4aed-8c94-49d8e76cc687 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:57.554354 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ce698-64a4-4aed-8c94-49d8e76cc687 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:57.575980 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1132/2284] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:30:57.645622 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ca3ca-d793-49eb-acec-690a1eb34869 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:57.646896 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ca3ca-d793-49eb-acec-690a1eb34869 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:57.646896 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ca3ca-d793-49eb-acec-690a1eb34869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:57.678080 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-059ca3ca-d793-49eb-acec-690a1eb34869 None None] MFA Rules not processed for user `f25c72896a2d4d619e5ad4b5a0a79aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:57.679373 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-059ca3ca-d793-49eb-acec-690a1eb34869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:57.783307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1153/2285] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:58.121982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ce698-64a4-4aed-8c94-49d8e76cc687 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:58.124357 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-810ce698-64a4-4aed-8c94-49d8e76cc687 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:58.153012 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ce698-64a4-4aed-8c94-49d8e76cc687 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:58.163507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea1998-c685-4b60-9c9e-8653b5b2593e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:58.164475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea1998-c685-4b60-9c9e-8653b5b2593e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:58.165293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea1998-c685-4b60-9c9e-8653b5b2593e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:58.165629 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea1998-c685-4b60-9c9e-8653b5b2593e admin admin] PATH_INFO: `/v3/projects/9044f5482f98459a93f1a186254196c0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:58.166922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea1998-c685-4b60-9c9e-8653b5b2593e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9044f5482f98459a93f1a186254196c0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:58.169643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea1998-c685-4b60-9c9e-8653b5b2593e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:58.255189 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7eea1998-c685-4b60-9c9e-8653b5b2593e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9044f5482f98459a93f1a186254196c0', 'request_id': 'req-7eea1998-c685-4b60-9c9e-8653b5b2593e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:58.261280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7eea1998-c685-4b60-9c9e-8653b5b2593e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9044f5482f98459a93f1a186254196c0', 'request_id': 'req-7eea1998-c685-4b60-9c9e-8653b5b2593e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:58.269071 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7eea1998-c685-4b60-9c9e-8653b5b2593e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9044f5482f98459a93f1a186254196c0', 'request_id': 'req-7eea1998-c685-4b60-9c9e-8653b5b2593e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:58.290079 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7eea1998-c685-4b60-9c9e-8653b5b2593e admin admin] The token cache is being invalidate because project 9044f5482f98459a93f1a186254196c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:30:58.290602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7eea1998-c685-4b60-9c9e-8653b5b2593e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:58.291317 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7eea1998-c685-4b60-9c9e-8653b5b2593e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:30:58.300968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1133/2286] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:30:58 2022] DELETE /identity/v3/projects/9044f5482f98459a93f1a186254196c0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:58.301270 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f942f2c-7f52-483d-b7ef-e1255a0cc30c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:58.301954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f942f2c-7f52-483d-b7ef-e1255a0cc30c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:58.301954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f942f2c-7f52-483d-b7ef-e1255a0cc30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:58.327085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3f942f2c-7f52-483d-b7ef-e1255a0cc30c None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:58.328860 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3f942f2c-7f52-483d-b7ef-e1255a0cc30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:58.418988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:58.446424 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05552579-3fa0-4a65-bdf7-fc5dedeccbc1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:58.446862 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05552579-3fa0-4a65-bdf7-fc5dedeccbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:58.447439 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05552579-3fa0-4a65-bdf7-fc5dedeccbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:58.456232 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1154/2287] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:30:58.472840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-05552579-3fa0-4a65-bdf7-fc5dedeccbc1 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:58.474474 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-05552579-3fa0-4a65-bdf7-fc5dedeccbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:58.493208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1134/2288] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:58.493549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f942f2c-7f52-483d-b7ef-e1255a0cc30c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:58.504275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f942f2c-7f52-483d-b7ef-e1255a0cc30c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:58.522753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54280ae0-0ae2-4b04-a945-16496543b9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:58.523521 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54280ae0-0ae2-4b04-a945-16496543b9ce admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:58.523999 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54280ae0-0ae2-4b04-a945-16496543b9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:58.524291 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54280ae0-0ae2-4b04-a945-16496543b9ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:58.525144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54280ae0-0ae2-4b04-a945-16496543b9ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:30:58.525463 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54280ae0-0ae2-4b04-a945-16496543b9ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:58.529677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54280ae0-0ae2-4b04-a945-16496543b9ce admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:58.563258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1155/2289] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:30:58.567444 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05552579-3fa0-4a65-bdf7-fc5dedeccbc1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:58.580969 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05552579-3fa0-4a65-bdf7-fc5dedeccbc1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:58.597172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b3412-33ea-4071-9b8e-737f333c67b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:58.597857 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b3412-33ea-4071-9b8e-737f333c67b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:58.598146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b3412-33ea-4071-9b8e-737f333c67b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:58.598426 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b3412-33ea-4071-9b8e-737f333c67b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:58.599148 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b3412-33ea-4071-9b8e-737f333c67b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:58.601246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b3412-33ea-4071-9b8e-737f333c67b9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:58.642317 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54280ae0-0ae2-4b04-a945-16496543b9ce admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:58.648999 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1135/2290] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:30:58 2022] POST /identity/v3/projects => generated 391 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:58.654711 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54280ae0-0ae2-4b04-a945-16496543b9ce admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:58.765291 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9cdca-f115-4c9c-99f3-eeb1dd281595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:58.766304 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9cdca-f115-4c9c-99f3-eeb1dd281595 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:58.766881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9cdca-f115-4c9c-99f3-eeb1dd281595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:58.767665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9cdca-f115-4c9c-99f3-eeb1dd281595 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:58.768561 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9cdca-f115-4c9c-99f3-eeb1dd281595 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:58.770548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9cdca-f115-4c9c-99f3-eeb1dd281595 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:58.819425 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1156/2291] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:30:58 2022] POST /identity/v3/users => generated 296 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:30:58.821686 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b3412-33ea-4071-9b8e-737f333c67b9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:58.828305 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:30:58.850622 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b3412-33ea-4071-9b8e-737f333c67b9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:58.861567 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76011e7b-9cec-49d5-832b-d9b95c49b445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:58.862873 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76011e7b-9cec-49d5-832b-d9b95c49b445 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:58.863166 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76011e7b-9cec-49d5-832b-d9b95c49b445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:58.865767 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76011e7b-9cec-49d5-832b-d9b95c49b445 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:58.866588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76011e7b-9cec-49d5-832b-d9b95c49b445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:58.869758 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76011e7b-9cec-49d5-832b-d9b95c49b445 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:58.894316 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1136/2292] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:58.898657 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9cdca-f115-4c9c-99f3-eeb1dd281595 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:58.914978 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9cdca-f115-4c9c-99f3-eeb1dd281595 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:58.930207 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c96a2-e859-4407-9628-45f9606c19c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:58.930893 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c96a2-e859-4407-9628-45f9606c19c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:58.931182 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c96a2-e859-4407-9628-45f9606c19c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:58.931462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c96a2-e859-4407-9628-45f9606c19c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:58.932207 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c96a2-e859-4407-9628-45f9606c19c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:58.933882 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c96a2-e859-4407-9628-45f9606c19c3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:58.952550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1157/2293] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:30:58.955898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76011e7b-9cec-49d5-832b-d9b95c49b445 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:58.977232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76011e7b-9cec-49d5-832b-d9b95c49b445 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:58.983362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d861c27-58f8-4298-bd06-ca4880654745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:58.984465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d861c27-58f8-4298-bd06-ca4880654745 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:58.985715 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d861c27-58f8-4298-bd06-ca4880654745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:58.986391 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d861c27-58f8-4298-bd06-ca4880654745 admin admin] PATH_INFO: `/v3/projects/01548fa710ae4d03ac718a818917935a/users/3a0872c09421469cb7c1665bb3993c08/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:59.006483 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d861c27-58f8-4298-bd06-ca4880654745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01548fa710ae4d03ac718a818917935a, user_id=3a0872c09421469cb7c1665bb3993c08, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:59.010614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d861c27-58f8-4298-bd06-ca4880654745 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:59.030476 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1137/2294] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:30:58 2022] PUT /identity/v3/projects/01548fa710ae4d03ac718a818917935a/users/3a0872c09421469cb7c1665bb3993c08/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:30:59.043008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acac42c-c4e1-4672-a375-52c472b0e392 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:59.043538 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acac42c-c4e1-4672-a375-52c472b0e392 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:59.043925 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acac42c-c4e1-4672-a375-52c472b0e392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:59.076405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5acac42c-c4e1-4672-a375-52c472b0e392 None None] MFA Rules not processed for user `3a0872c09421469cb7c1665bb3993c08`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:30:59.078802 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5acac42c-c4e1-4672-a375-52c472b0e392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:59.192959 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1158/2295] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:30:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:30:59.205471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d861c27-58f8-4298-bd06-ca4880654745 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:30:59.208278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9d861c27-58f8-4298-bd06-ca4880654745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:30:59.287285 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d861c27-58f8-4298-bd06-ca4880654745 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:30:59.293931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a233c10-996b-413a-ad9f-610a00383e74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:30:59.295250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a233c10-996b-413a-ad9f-610a00383e74 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:30:59.295680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a233c10-996b-413a-ad9f-610a00383e74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:30:59.296036 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a233c10-996b-413a-ad9f-610a00383e74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:30:59.307165 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a233c10-996b-413a-ad9f-610a00383e74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:30:59.310449 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a233c10-996b-413a-ad9f-610a00383e74 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:30:59.327960 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1138/2296] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:30:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:31:03.049828 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0626a6f-becb-4918-b0a6-eb1b2a17ef7a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:03.050428 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0626a6f-becb-4918-b0a6-eb1b2a17ef7a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:03.050428 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0626a6f-becb-4918-b0a6-eb1b2a17ef7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:03.070616 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a0626a6f-becb-4918-b0a6-eb1b2a17ef7a None None] MFA Rules not processed for user `72a431c9708f4694bc0d514133e76088`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:03.071860 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a0626a6f-becb-4918-b0a6-eb1b2a17ef7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:03.132190 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:31:03.148857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1159/2297] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:31:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:31:09.319322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:31:09.937218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8ce61-afc5-4c32-990d-0e37f6b54084 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:09.943741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8ce61-afc5-4c32-990d-0e37f6b54084 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:09.944327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8ce61-afc5-4c32-990d-0e37f6b54084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:09.966216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f9e8ce61-afc5-4c32-990d-0e37f6b54084 None None] MFA Rules not processed for user `a09029144c454b099bea86aaa5229ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:09.967964 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f9e8ce61-afc5-4c32-990d-0e37f6b54084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:10.040983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1139/2298] 10.222.0.33 () {58 vars in 967 bytes} [Mon Jul 18 10:31:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:31:10.057626 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0626a6f-becb-4918-b0a6-eb1b2a17ef7a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:10.059904 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a0626a6f-becb-4918-b0a6-eb1b2a17ef7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:10.155321 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0626a6f-becb-4918-b0a6-eb1b2a17ef7a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:10.160314 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aca721-e8d5-457c-ad6c-11f8738ecfc3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6168873f2846c8af73b903bba17eb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:10.160989 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aca721-e8d5-457c-ad6c-11f8738ecfc3 service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:10.161274 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aca721-e8d5-457c-ad6c-11f8738ecfc3 service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:10.161553 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aca721-e8d5-457c-ad6c-11f8738ecfc3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:10.169683 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aca721-e8d5-457c-ad6c-11f8738ecfc3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:10.171265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aca721-e8d5-457c-ad6c-11f8738ecfc3 service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:10.179554 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1160/2299] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 18 10:31:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:31:14.294982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8ce61-afc5-4c32-990d-0e37f6b54084 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:14.296671 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f9e8ce61-afc5-4c32-990d-0e37f6b54084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:14.401680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8ce61-afc5-4c32-990d-0e37f6b54084 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:14.407413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc40e9f-33a0-454e-b480-dcbcc06b118d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:14.408312 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc40e9f-33a0-454e-b480-dcbcc06b118d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:14.408655 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc40e9f-33a0-454e-b480-dcbcc06b118d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:14.408937 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc40e9f-33a0-454e-b480-dcbcc06b118d admin admin] PATH_INFO: `/v3/users/bc233897770f47e5bd40fb03afabe49c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:14.410278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc40e9f-33a0-454e-b480-dcbcc06b118d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc233897770f47e5bd40fb03afabe49c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:14.412256 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc40e9f-33a0-454e-b480-dcbcc06b118d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:14.523769 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5dc40e9f-33a0-454e-b480-dcbcc06b118d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc233897770f47e5bd40fb03afabe49c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:14.525337 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5dc40e9f-33a0-454e-b480-dcbcc06b118d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc233897770f47e5bd40fb03afabe49c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:14.525850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5dc40e9f-33a0-454e-b480-dcbcc06b118d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc233897770f47e5bd40fb03afabe49c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:14.534042 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5dc40e9f-33a0-454e-b480-dcbcc06b118d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc233897770f47e5bd40fb03afabe49c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:14.558510 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5dc40e9f-33a0-454e-b480-dcbcc06b118d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc233897770f47e5bd40fb03afabe49c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:14.575026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1140/2300] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:31:14 2022] DELETE /identity/v3/users/bc233897770f47e5bd40fb03afabe49c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:31:14.909491 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aca721-e8d5-457c-ad6c-11f8738ecfc3 service swift] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:14.932749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aca721-e8d5-457c-ad6c-11f8738ecfc3 service swift] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:15.030985 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a59d3-2be4-4711-b7ba-95dd09fb7982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:15.032799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a59d3-2be4-4711-b7ba-95dd09fb7982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:15.033149 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a59d3-2be4-4711-b7ba-95dd09fb7982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:15.033441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a59d3-2be4-4711-b7ba-95dd09fb7982 admin admin] PATH_INFO: `/v3/projects/9698a0142f494095a93a4e804437bc32` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:15.034702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7a59d3-2be4-4711-b7ba-95dd09fb7982 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9698a0142f494095a93a4e804437bc32)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:15.036745 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7a59d3-2be4-4711-b7ba-95dd09fb7982 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:15.092043 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1f7a59d3-2be4-4711-b7ba-95dd09fb7982 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9698a0142f494095a93a4e804437bc32', 'request_id': 'req-1f7a59d3-2be4-4711-b7ba-95dd09fb7982'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:15.098656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1f7a59d3-2be4-4711-b7ba-95dd09fb7982 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9698a0142f494095a93a4e804437bc32', 'request_id': 'req-1f7a59d3-2be4-4711-b7ba-95dd09fb7982'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:15.105804 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1f7a59d3-2be4-4711-b7ba-95dd09fb7982 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9698a0142f494095a93a4e804437bc32', 'request_id': 'req-1f7a59d3-2be4-4711-b7ba-95dd09fb7982'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:15.118670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1f7a59d3-2be4-4711-b7ba-95dd09fb7982 admin admin] The token cache is being invalidate because project 9698a0142f494095a93a4e804437bc32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:31:15.119184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1f7a59d3-2be4-4711-b7ba-95dd09fb7982 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:15.119964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1f7a59d3-2be4-4711-b7ba-95dd09fb7982 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:15.121589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1161/2301] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:31:14 2022] DELETE /identity/v3/projects/9698a0142f494095a93a4e804437bc32 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:31:15.126641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaeee2-d20d-4201-bbda-40064cced093 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:15.126938 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaeee2-d20d-4201-bbda-40064cced093 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:15.127244 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaeee2-d20d-4201-bbda-40064cced093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:15.153310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-79aaeee2-d20d-4201-bbda-40064cced093 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:15.154892 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-79aaeee2-d20d-4201-bbda-40064cced093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:15.176736 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:31:15.225288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1141/2302] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:31:15.233483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bb99a-5ac9-4a38-b66b-54cee2aa4699 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:15.233849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bb99a-5ac9-4a38-b66b-54cee2aa4699 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:15.234140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bb99a-5ac9-4a38-b66b-54cee2aa4699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:15.257559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-884bb99a-5ac9-4a38-b66b-54cee2aa4699 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:15.258943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-884bb99a-5ac9-4a38-b66b-54cee2aa4699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:15.268623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1162/2303] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:31:15.272906 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaeee2-d20d-4201-bbda-40064cced093 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:15.282800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaeee2-d20d-4201-bbda-40064cced093 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:15.287524 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a8c9b-9bb9-42a7-ac2a-c05095b3fa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:15.288187 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a8c9b-9bb9-42a7-ac2a-c05095b3fa39 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:15.288475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a8c9b-9bb9-42a7-ac2a-c05095b3fa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:15.288784 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a8c9b-9bb9-42a7-ac2a-c05095b3fa39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:15.289520 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a8c9b-9bb9-42a7-ac2a-c05095b3fa39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:31:15.289829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a8c9b-9bb9-42a7-ac2a-c05095b3fa39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:15.291430 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a8c9b-9bb9-42a7-ac2a-c05095b3fa39 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:15.307000 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1142/2304] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:31:15.308862 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bb99a-5ac9-4a38-b66b-54cee2aa4699 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:15.318724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bb99a-5ac9-4a38-b66b-54cee2aa4699 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:15.324749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab112f33-2f6f-41eb-8fdd-ae6e40b47dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:15.325423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab112f33-2f6f-41eb-8fdd-ae6e40b47dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:15.325709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab112f33-2f6f-41eb-8fdd-ae6e40b47dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:15.325986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab112f33-2f6f-41eb-8fdd-ae6e40b47dd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:15.326685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab112f33-2f6f-41eb-8fdd-ae6e40b47dd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:15.329245 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab112f33-2f6f-41eb-8fdd-ae6e40b47dd0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:15.333712 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:31:15.362291 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1163/2305] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:31:15 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:31:15.363602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a8c9b-9bb9-42a7-ac2a-c05095b3fa39 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:15.373843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a8c9b-9bb9-42a7-ac2a-c05095b3fa39 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:15.430913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef936ae-d6df-4ed2-a268-9c9f145bf1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:15.431578 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef936ae-d6df-4ed2-a268-9c9f145bf1b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:15.431865 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef936ae-d6df-4ed2-a268-9c9f145bf1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:15.432142 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef936ae-d6df-4ed2-a268-9c9f145bf1b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:15.432863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef936ae-d6df-4ed2-a268-9c9f145bf1b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:15.434954 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef936ae-d6df-4ed2-a268-9c9f145bf1b6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:15.458233 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1143/2306] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:31:15 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:31:15.460945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab112f33-2f6f-41eb-8fdd-ae6e40b47dd0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:15.471052 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab112f33-2f6f-41eb-8fdd-ae6e40b47dd0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:15.475825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd8397-b713-4ed0-a4bc-8da85538b8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:15.476424 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd8397-b713-4ed0-a4bc-8da85538b8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:15.476786 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd8397-b713-4ed0-a4bc-8da85538b8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:15.477070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd8397-b713-4ed0-a4bc-8da85538b8b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:15.477833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd8397-b713-4ed0-a4bc-8da85538b8b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:15.479707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd8397-b713-4ed0-a4bc-8da85538b8b2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:15.489656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1164/2307] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:31:15.493457 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef936ae-d6df-4ed2-a268-9c9f145bf1b6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:15.503931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef936ae-d6df-4ed2-a268-9c9f145bf1b6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:15.508604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9014b-53a2-4a13-8fb8-a1e6d08620a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:15.509252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9014b-53a2-4a13-8fb8-a1e6d08620a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:15.509555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9014b-53a2-4a13-8fb8-a1e6d08620a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:15.509856 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9014b-53a2-4a13-8fb8-a1e6d08620a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:15.510573 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b9014b-53a2-4a13-8fb8-a1e6d08620a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:15.512146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b9014b-53a2-4a13-8fb8-a1e6d08620a3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:15.522583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1144/2308] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:31:15.526691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd8397-b713-4ed0-a4bc-8da85538b8b2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:15.536636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd8397-b713-4ed0-a4bc-8da85538b8b2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:15.541276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53addbe-1648-4502-9159-a7149ada0752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:15.542150 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53addbe-1648-4502-9159-a7149ada0752 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:15.542432 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53addbe-1648-4502-9159-a7149ada0752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:15.542712 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53addbe-1648-4502-9159-a7149ada0752 admin admin] PATH_INFO: `/v3/projects/4fff06a684454873a9e897d80705d456/users/75e0ecd071384e708eb22cb6ac47923d/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:15.557758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53addbe-1648-4502-9159-a7149ada0752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fff06a684454873a9e897d80705d456, user_id=75e0ecd071384e708eb22cb6ac47923d, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:15.560391 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53addbe-1648-4502-9159-a7149ada0752 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:15.573876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1165/2309] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:31:15 2022] PUT /identity/v3/projects/4fff06a684454873a9e897d80705d456/users/75e0ecd071384e708eb22cb6ac47923d/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:31:15.584627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2856e-0424-420d-8bbb-4375543f162c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:15.584924 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2856e-0424-420d-8bbb-4375543f162c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:15.585208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2856e-0424-420d-8bbb-4375543f162c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:15.612434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-54c2856e-0424-420d-8bbb-4375543f162c None None] MFA Rules not processed for user `75e0ecd071384e708eb22cb6ac47923d`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:15.613674 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-54c2856e-0424-420d-8bbb-4375543f162c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:15.682311 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1145/2310] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:31:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:31:15.693444 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53addbe-1648-4502-9159-a7149ada0752 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:15.695126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a53addbe-1648-4502-9159-a7149ada0752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:15.752425 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53addbe-1648-4502-9159-a7149ada0752 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:15.756098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f20fe-8786-46d4-841c-31ab59b084d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:15.756767 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f20fe-8786-46d4-841c-31ab59b084d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:15.757053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f20fe-8786-46d4-841c-31ab59b084d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:15.757331 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f20fe-8786-46d4-841c-31ab59b084d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:15.766664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f20fe-8786-46d4-841c-31ab59b084d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:15.768161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f20fe-8786-46d4-841c-31ab59b084d7 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:15.777865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1166/2311] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:31:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:31:24.609378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2856e-0424-420d-8bbb-4375543f162c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:24.611310 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-54c2856e-0424-420d-8bbb-4375543f162c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:24.633097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:31:24.676810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2856e-0424-420d-8bbb-4375543f162c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:24.681380 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9fc2a-3410-4ba3-abd8-ff3c52c8e7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:24.682260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9fc2a-3410-4ba3-abd8-ff3c52c8e7f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:24.682544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9fc2a-3410-4ba3-abd8-ff3c52c8e7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:24.683076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9fc2a-3410-4ba3-abd8-ff3c52c8e7f0 admin admin] PATH_INFO: `/v3/users/3a0872c09421469cb7c1665bb3993c08` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:24.684489 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9fc2a-3410-4ba3-abd8-ff3c52c8e7f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a0872c09421469cb7c1665bb3993c08)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:24.686668 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9fc2a-3410-4ba3-abd8-ff3c52c8e7f0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:24.733058 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fcb9fc2a-3410-4ba3-abd8-ff3c52c8e7f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a0872c09421469cb7c1665bb3993c08'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:24.733706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fcb9fc2a-3410-4ba3-abd8-ff3c52c8e7f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a0872c09421469cb7c1665bb3993c08'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:24.734301 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fcb9fc2a-3410-4ba3-abd8-ff3c52c8e7f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a0872c09421469cb7c1665bb3993c08'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:24.741711 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fcb9fc2a-3410-4ba3-abd8-ff3c52c8e7f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a0872c09421469cb7c1665bb3993c08'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:24.754486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fcb9fc2a-3410-4ba3-abd8-ff3c52c8e7f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a0872c09421469cb7c1665bb3993c08'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:24.767536 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1146/2312] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:31:24 2022] DELETE /identity/v3/users/3a0872c09421469cb7c1665bb3993c08 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:31:24.772205 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e4cef-2a85-4606-8691-03952e017170 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:24.772720 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e4cef-2a85-4606-8691-03952e017170 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:24.772835 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e4cef-2a85-4606-8691-03952e017170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:24.789565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ea4e4cef-2a85-4606-8691-03952e017170 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:24.790753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ea4e4cef-2a85-4606-8691-03952e017170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:24.848217 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1167/2313] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:31:24.858461 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9fc2a-3410-4ba3-abd8-ff3c52c8e7f0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:24.859871 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fcb9fc2a-3410-4ba3-abd8-ff3c52c8e7f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:24.919807 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9fc2a-3410-4ba3-abd8-ff3c52c8e7f0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:24.923680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73cb52-6608-45aa-baff-413e2baff70e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:24.924336 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73cb52-6608-45aa-baff-413e2baff70e service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:24.924658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73cb52-6608-45aa-baff-413e2baff70e service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:24.924966 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73cb52-6608-45aa-baff-413e2baff70e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:24.934550 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73cb52-6608-45aa-baff-413e2baff70e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:24.935987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73cb52-6608-45aa-baff-413e2baff70e service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:24.946931 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1147/2314] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:31:25.374820 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e4cef-2a85-4606-8691-03952e017170 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:25.376913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ea4e4cef-2a85-4606-8691-03952e017170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:25.389811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e4cef-2a85-4606-8691-03952e017170 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:25.394683 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c5e5c-15bd-4ddd-bab7-30ff2e4147e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:25.395545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c5e5c-15bd-4ddd-bab7-30ff2e4147e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:25.395829 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c5e5c-15bd-4ddd-bab7-30ff2e4147e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:25.396110 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c5e5c-15bd-4ddd-bab7-30ff2e4147e8 admin admin] PATH_INFO: `/v3/projects/01548fa710ae4d03ac718a818917935a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:25.397323 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c5e5c-15bd-4ddd-bab7-30ff2e4147e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01548fa710ae4d03ac718a818917935a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:25.399390 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c5e5c-15bd-4ddd-bab7-30ff2e4147e8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:25.458578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a12c5e5c-15bd-4ddd-bab7-30ff2e4147e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01548fa710ae4d03ac718a818917935a', 'request_id': 'req-a12c5e5c-15bd-4ddd-bab7-30ff2e4147e8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:25.460422 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:31:25.466108 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a12c5e5c-15bd-4ddd-bab7-30ff2e4147e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01548fa710ae4d03ac718a818917935a', 'request_id': 'req-a12c5e5c-15bd-4ddd-bab7-30ff2e4147e8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:25.476925 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a12c5e5c-15bd-4ddd-bab7-30ff2e4147e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01548fa710ae4d03ac718a818917935a', 'request_id': 'req-a12c5e5c-15bd-4ddd-bab7-30ff2e4147e8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:25.492300 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a12c5e5c-15bd-4ddd-bab7-30ff2e4147e8 admin admin] The token cache is being invalidate because project 01548fa710ae4d03ac718a818917935a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:31:25.493024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a12c5e5c-15bd-4ddd-bab7-30ff2e4147e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:25.493688 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a12c5e5c-15bd-4ddd-bab7-30ff2e4147e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:25.496559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1168/2315] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:31:25 2022] DELETE /identity/v3/projects/01548fa710ae4d03ac718a818917935a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:31:25.500559 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749058f7-fedd-485c-910b-2a245d79cae8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:25.501080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749058f7-fedd-485c-910b-2a245d79cae8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:25.501939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749058f7-fedd-485c-910b-2a245d79cae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:25.520472 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-749058f7-fedd-485c-910b-2a245d79cae8 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:25.521804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-749058f7-fedd-485c-910b-2a245d79cae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:25.560783 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:31:25.590787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1148/2316] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:31:25.598920 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098a7a2-b2d3-4172-826c-dfb44cc6a58e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:25.599219 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098a7a2-b2d3-4172-826c-dfb44cc6a58e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:25.599493 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098a7a2-b2d3-4172-826c-dfb44cc6a58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:25.619608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1098a7a2-b2d3-4172-826c-dfb44cc6a58e None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:25.620993 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1098a7a2-b2d3-4172-826c-dfb44cc6a58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:25.629032 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1169/2317] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:31:25.633345 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749058f7-fedd-485c-910b-2a245d79cae8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:25.643914 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749058f7-fedd-485c-910b-2a245d79cae8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:25.648740 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7ae86-ce61-442c-a11b-fed5a75bac3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:25.649404 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7ae86-ce61-442c-a11b-fed5a75bac3d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:25.649687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7ae86-ce61-442c-a11b-fed5a75bac3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:25.649963 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7ae86-ce61-442c-a11b-fed5a75bac3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:25.650718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7ae86-ce61-442c-a11b-fed5a75bac3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:31:25.651026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7ae86-ce61-442c-a11b-fed5a75bac3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:25.652632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7ae86-ce61-442c-a11b-fed5a75bac3d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:25.668538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1149/2318] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:31:25.669990 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098a7a2-b2d3-4172-826c-dfb44cc6a58e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:25.679531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098a7a2-b2d3-4172-826c-dfb44cc6a58e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:25.684294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e80567-dddc-4584-b6d2-2c2d119a4f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:25.685033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e80567-dddc-4584-b6d2-2c2d119a4f2a admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:25.685319 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e80567-dddc-4584-b6d2-2c2d119a4f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:25.685594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e80567-dddc-4584-b6d2-2c2d119a4f2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:25.686329 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e80567-dddc-4584-b6d2-2c2d119a4f2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:25.688321 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e80567-dddc-4584-b6d2-2c2d119a4f2a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:25.716880 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1170/2319] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:31:25 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:31:25.718120 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7ae86-ce61-442c-a11b-fed5a75bac3d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:25.732790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7ae86-ce61-442c-a11b-fed5a75bac3d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:25.792031 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ae0aa-3477-4644-8512-0b58517439ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:25.792806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ae0aa-3477-4644-8512-0b58517439ec admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:25.793091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ae0aa-3477-4644-8512-0b58517439ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:25.793367 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ae0aa-3477-4644-8512-0b58517439ec admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:25.794058 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ae0aa-3477-4644-8512-0b58517439ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:25.796010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ae0aa-3477-4644-8512-0b58517439ec admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:25.820086 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1150/2320] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:31:25 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:31:25.821112 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e80567-dddc-4584-b6d2-2c2d119a4f2a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:25.832718 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e80567-dddc-4584-b6d2-2c2d119a4f2a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:25.837512 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f02b3-cbab-4bbe-8d79-d7a044a865fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:25.838249 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f02b3-cbab-4bbe-8d79-d7a044a865fe admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:25.838610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f02b3-cbab-4bbe-8d79-d7a044a865fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:25.838964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f02b3-cbab-4bbe-8d79-d7a044a865fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:25.839705 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f02b3-cbab-4bbe-8d79-d7a044a865fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:25.841375 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f02b3-cbab-4bbe-8d79-d7a044a865fe admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:25.852077 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1171/2321] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:31:25.856275 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ae0aa-3477-4644-8512-0b58517439ec admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:25.866733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ae0aa-3477-4644-8512-0b58517439ec admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:25.872311 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed1dec-3bce-4d25-b462-8c877c86ec83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:25.872942 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed1dec-3bce-4d25-b462-8c877c86ec83 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:25.873236 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed1dec-3bce-4d25-b462-8c877c86ec83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:25.873521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed1dec-3bce-4d25-b462-8c877c86ec83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:25.874251 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed1dec-3bce-4d25-b462-8c877c86ec83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:25.875868 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed1dec-3bce-4d25-b462-8c877c86ec83 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:25.886518 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1151/2322] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:31:25.890795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f02b3-cbab-4bbe-8d79-d7a044a865fe admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:25.901460 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f02b3-cbab-4bbe-8d79-d7a044a865fe admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:25.909885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f66b3-4886-4f3b-b97a-9b08816a7b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:25.910798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f66b3-4886-4f3b-b97a-9b08816a7b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:25.911087 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f66b3-4886-4f3b-b97a-9b08816a7b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:25.911367 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f66b3-4886-4f3b-b97a-9b08816a7b00 admin admin] PATH_INFO: `/v3/projects/f62ef510c8484995a2f31f29104f399f/users/1af30265de71409d876112350949a2a4/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:25.927623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f66b3-4886-4f3b-b97a-9b08816a7b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f62ef510c8484995a2f31f29104f399f, user_id=1af30265de71409d876112350949a2a4, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:25.930544 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f66b3-4886-4f3b-b97a-9b08816a7b00 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:25.943751 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1172/2323] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:31:25 2022] PUT /identity/v3/projects/f62ef510c8484995a2f31f29104f399f/users/1af30265de71409d876112350949a2a4/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:31:25.948907 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5712a06-3c78-4088-b429-10049e4fe974 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:25.949201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5712a06-3c78-4088-b429-10049e4fe974 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:25.949482 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5712a06-3c78-4088-b429-10049e4fe974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:25.967611 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b5712a06-3c78-4088-b429-10049e4fe974 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:25.968839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b5712a06-3c78-4088-b429-10049e4fe974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:26.030455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1152/2324] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:31:26.041360 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f66b3-4886-4f3b-b97a-9b08816a7b00 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:26.043192 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-346f66b3-4886-4f3b-b97a-9b08816a7b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:26.101647 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f66b3-4886-4f3b-b97a-9b08816a7b00 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:26.105496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57bfe82-d256-4486-ad13-0c1930860d16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:26.106144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bfe82-d256-4486-ad13-0c1930860d16 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:26.106430 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bfe82-d256-4486-ad13-0c1930860d16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:26.106705 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57bfe82-d256-4486-ad13-0c1930860d16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:26.116132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57bfe82-d256-4486-ad13-0c1930860d16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:26.117587 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57bfe82-d256-4486-ad13-0c1930860d16 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:26.128011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1173/2325] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:31:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:31:31.372868 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5712a06-3c78-4088-b429-10049e4fe974 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:31.375150 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b5712a06-3c78-4088-b429-10049e4fe974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:31.405121 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5712a06-3c78-4088-b429-10049e4fe974 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:31.414352 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e3ea3-94dd-4845-ab07-29b5025deaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:31.415275 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e3ea3-94dd-4845-ab07-29b5025deaf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:31.415621 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e3ea3-94dd-4845-ab07-29b5025deaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:31.416173 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e3ea3-94dd-4845-ab07-29b5025deaf4 admin admin] PATH_INFO: `/v3/users/75e0ecd071384e708eb22cb6ac47923d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:31.417574 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e3ea3-94dd-4845-ab07-29b5025deaf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75e0ecd071384e708eb22cb6ac47923d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:31.420019 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e3ea3-94dd-4845-ab07-29b5025deaf4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:31.491214 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:31:31.500682 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb6e3ea3-94dd-4845-ab07-29b5025deaf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75e0ecd071384e708eb22cb6ac47923d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:31.501420 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb6e3ea3-94dd-4845-ab07-29b5025deaf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75e0ecd071384e708eb22cb6ac47923d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:31.502060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb6e3ea3-94dd-4845-ab07-29b5025deaf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75e0ecd071384e708eb22cb6ac47923d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:31.512354 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb6e3ea3-94dd-4845-ab07-29b5025deaf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75e0ecd071384e708eb22cb6ac47923d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:31.535688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb6e3ea3-94dd-4845-ab07-29b5025deaf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75e0ecd071384e708eb22cb6ac47923d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:31.553175 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1153/2326] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:31:31 2022] DELETE /identity/v3/users/75e0ecd071384e708eb22cb6ac47923d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:31:31.557964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133e922-2721-49ae-83ad-9375d1d88f91 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:31.558498 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133e922-2721-49ae-83ad-9375d1d88f91 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:31.559201 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133e922-2721-49ae-83ad-9375d1d88f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:31.594614 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3133e922-2721-49ae-83ad-9375d1d88f91 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:31.601300 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3133e922-2721-49ae-83ad-9375d1d88f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:31.740587 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1174/2327] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:31:31.758167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e3ea3-94dd-4845-ab07-29b5025deaf4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:31.759593 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cb6e3ea3-94dd-4845-ab07-29b5025deaf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:31.886060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e3ea3-94dd-4845-ab07-29b5025deaf4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:31.894730 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734a4d6-8b0f-4fea-9c1f-b81e755f7761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:31.895552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734a4d6-8b0f-4fea-9c1f-b81e755f7761 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:31.896980 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734a4d6-8b0f-4fea-9c1f-b81e755f7761 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:31.897134 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734a4d6-8b0f-4fea-9c1f-b81e755f7761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:31.928539 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734a4d6-8b0f-4fea-9c1f-b81e755f7761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:31.930363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734a4d6-8b0f-4fea-9c1f-b81e755f7761 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:31.961056 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1154/2328] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:31:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:31:32.608192 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55412a-bb42-46eb-8ce8-f0b96a23a9e8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:32.608730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55412a-bb42-46eb-8ce8-f0b96a23a9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:32.609097 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55412a-bb42-46eb-8ce8-f0b96a23a9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:32.694540 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0a55412a-bb42-46eb-8ce8-f0b96a23a9e8 None None] MFA Rules not processed for user `1af30265de71409d876112350949a2a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:32.698427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0a55412a-bb42-46eb-8ce8-f0b96a23a9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:32.722503 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:31:32.811657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734a4d6-8b0f-4fea-9c1f-b81e755f7761 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:32.816130 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5734a4d6-8b0f-4fea-9c1f-b81e755f7761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:32.836302 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734a4d6-8b0f-4fea-9c1f-b81e755f7761 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:32.841799 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056c7ad-527d-4702-84d2-bf8ad803b8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:32.842670 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056c7ad-527d-4702-84d2-bf8ad803b8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:32.843350 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056c7ad-527d-4702-84d2-bf8ad803b8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:32.843646 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056c7ad-527d-4702-84d2-bf8ad803b8e1 admin admin] PATH_INFO: `/v3/projects/4fff06a684454873a9e897d80705d456` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:32.844945 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056c7ad-527d-4702-84d2-bf8ad803b8e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fff06a684454873a9e897d80705d456)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:32.848586 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056c7ad-527d-4702-84d2-bf8ad803b8e1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:32.854371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1175/2329] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:31:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:31:32.860897 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55412a-bb42-46eb-8ce8-f0b96a23a9e8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:32.862969 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0a55412a-bb42-46eb-8ce8-f0b96a23a9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:32.898056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55412a-bb42-46eb-8ce8-f0b96a23a9e8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:32.903912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca1c4e-9e9d-4d0b-b22a-4eede3b0022b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:32.904653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca1c4e-9e9d-4d0b-b22a-4eede3b0022b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:32.904942 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca1c4e-9e9d-4d0b-b22a-4eede3b0022b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:32.905371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca1c4e-9e9d-4d0b-b22a-4eede3b0022b admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:32.906240 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ca1c4e-9e9d-4d0b-b22a-4eede3b0022b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:32.908457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ca1c4e-9e9d-4d0b-b22a-4eede3b0022b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:32.966507 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1176/2330] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:31:32 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:31:32.971550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca1c4e-9e9d-4d0b-b22a-4eede3b0022b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:33.091490 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca1c4e-9e9d-4d0b-b22a-4eede3b0022b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:33.111876 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c056c7ad-527d-4702-84d2-bf8ad803b8e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fff06a684454873a9e897d80705d456', 'request_id': 'req-c056c7ad-527d-4702-84d2-bf8ad803b8e1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:33.132995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c056c7ad-527d-4702-84d2-bf8ad803b8e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fff06a684454873a9e897d80705d456', 'request_id': 'req-c056c7ad-527d-4702-84d2-bf8ad803b8e1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:33.145694 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c056c7ad-527d-4702-84d2-bf8ad803b8e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fff06a684454873a9e897d80705d456', 'request_id': 'req-c056c7ad-527d-4702-84d2-bf8ad803b8e1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:33.172732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c056c7ad-527d-4702-84d2-bf8ad803b8e1 admin admin] The token cache is being invalidate because project 4fff06a684454873a9e897d80705d456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:31:33.173629 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c056c7ad-527d-4702-84d2-bf8ad803b8e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:33.174850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c056c7ad-527d-4702-84d2-bf8ad803b8e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:31:33.181234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1155/2331] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:31:32 2022] DELETE /identity/v3/projects/4fff06a684454873a9e897d80705d456 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:31:33.183441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96f223-56cf-48f1-950e-eee9a8d00cd2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:33.183582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96f223-56cf-48f1-950e-eee9a8d00cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:33.183979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96f223-56cf-48f1-950e-eee9a8d00cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:33.208172 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9490e1-42d2-4077-9810-eda54fc5d15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:33.208992 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9490e1-42d2-4077-9810-eda54fc5d15c admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:33.209348 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9490e1-42d2-4077-9810-eda54fc5d15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:33.210329 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9490e1-42d2-4077-9810-eda54fc5d15c admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:33.212369 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9490e1-42d2-4077-9810-eda54fc5d15c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:33.215830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9490e1-42d2-4077-9810-eda54fc5d15c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:33.226801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7f96f223-56cf-48f1-950e-eee9a8d00cd2 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:33.229111 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7f96f223-56cf-48f1-950e-eee9a8d00cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:33.244558 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1156/2332] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:31:33.250999 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffdb22-ce19-40ca-b947-eba14eeabc98 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:33.251639 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffdb22-ce19-40ca-b947-eba14eeabc98 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:33.251999 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffdb22-ce19-40ca-b947-eba14eeabc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:33.256493 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1177/2333] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:31:32 2022] POST /identity/v3/users => generated 299 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:31:33.260142 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9490e1-42d2-4077-9810-eda54fc5d15c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:33.266587 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7b9490e1-42d2-4077-9810-eda54fc5d15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:33.293970 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-41ffdb22-ce19-40ca-b947-eba14eeabc98 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:33.295319 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-41ffdb22-ce19-40ca-b947-eba14eeabc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:33.305299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9490e1-42d2-4077-9810-eda54fc5d15c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:33.305624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1157/2334] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:31:33.309673 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffdb22-ce19-40ca-b947-eba14eeabc98 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:33.311974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7904f5f-3db4-4c0b-9a36-fe57f622ece6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:33.313201 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7904f5f-3db4-4c0b-9a36-fe57f622ece6 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:33.313709 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7904f5f-3db4-4c0b-9a36-fe57f622ece6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:33.314172 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7904f5f-3db4-4c0b-9a36-fe57f622ece6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:33.315105 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7904f5f-3db4-4c0b-9a36-fe57f622ece6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:33.316962 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7904f5f-3db4-4c0b-9a36-fe57f622ece6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:33.319652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffdb22-ce19-40ca-b947-eba14eeabc98 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:33.324563 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50b26a-b9a8-4045-acae-86f88fe47f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:33.325234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50b26a-b9a8-4045-acae-86f88fe47f3d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:33.325570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50b26a-b9a8-4045-acae-86f88fe47f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:33.325851 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50b26a-b9a8-4045-acae-86f88fe47f3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:33.326589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50b26a-b9a8-4045-acae-86f88fe47f3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:31:33.326903 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50b26a-b9a8-4045-acae-86f88fe47f3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:33.328526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50b26a-b9a8-4045-acae-86f88fe47f3d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:33.341772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1158/2335] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:31:33.346324 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50b26a-b9a8-4045-acae-86f88fe47f3d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:33.347214 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1178/2336] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:31:33.351429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7904f5f-3db4-4c0b-9a36-fe57f622ece6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:33.369907 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50b26a-b9a8-4045-acae-86f88fe47f3d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:33.372079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7904f5f-3db4-4c0b-9a36-fe57f622ece6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:33.377808 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122dea9-166a-4fe4-935c-5aa7809c3975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:33.378691 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122dea9-166a-4fe4-935c-5aa7809c3975 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:33.379043 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122dea9-166a-4fe4-935c-5aa7809c3975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:33.379334 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122dea9-166a-4fe4-935c-5aa7809c3975 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:33.380147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122dea9-166a-4fe4-935c-5aa7809c3975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:33.380608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34317f90-ce79-41a4-a0b3-19422d01c49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:33.381931 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34317f90-ce79-41a4-a0b3-19422d01c49f admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:33.382563 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34317f90-ce79-41a4-a0b3-19422d01c49f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:33.383028 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34317f90-ce79-41a4-a0b3-19422d01c49f admin admin] PATH_INFO: `/v3/projects/ddd22b027e204f30848e177ea4b8dfe6/users/a2fc697f771d43b6a0f29f5adf8cb4d9/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:33.383157 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122dea9-166a-4fe4-935c-5aa7809c3975 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:33.405308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:31:33.425205 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34317f90-ce79-41a4-a0b3-19422d01c49f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddd22b027e204f30848e177ea4b8dfe6, user_id=a2fc697f771d43b6a0f29f5adf8cb4d9, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:33.430855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:31:33.433270 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34317f90-ce79-41a4-a0b3-19422d01c49f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:33.442331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1159/2337] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:31:33 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:31:33.446589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122dea9-166a-4fe4-935c-5aa7809c3975 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:33.459140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1179/2338] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:31:33 2022] PUT /identity/v3/projects/ddd22b027e204f30848e177ea4b8dfe6/users/a2fc697f771d43b6a0f29f5adf8cb4d9/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:31:33.463629 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34317f90-ce79-41a4-a0b3-19422d01c49f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:33.559258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34317f90-ce79-41a4-a0b3-19422d01c49f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:33.568468 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5db395-64db-43bc-9bbd-8e2c86eae3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:33.570479 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5db395-64db-43bc-9bbd-8e2c86eae3d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:33.571091 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5db395-64db-43bc-9bbd-8e2c86eae3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:33.576864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5db395-64db-43bc-9bbd-8e2c86eae3d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:33.576864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5db395-64db-43bc-9bbd-8e2c86eae3d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:33.576864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5db395-64db-43bc-9bbd-8e2c86eae3d4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:33.606924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1180/2339] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:31:33.607476 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122dea9-166a-4fe4-935c-5aa7809c3975 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:33.608020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5db395-64db-43bc-9bbd-8e2c86eae3d4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:33.617741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b618b82-121b-4b60-b68b-8feb361d0d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:33.618645 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b618b82-121b-4b60-b68b-8feb361d0d4d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:33.619220 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b618b82-121b-4b60-b68b-8feb361d0d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:33.619706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b618b82-121b-4b60-b68b-8feb361d0d4d admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:33.620619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b618b82-121b-4b60-b68b-8feb361d0d4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:33.623377 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b618b82-121b-4b60-b68b-8feb361d0d4d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:33.625990 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5db395-64db-43bc-9bbd-8e2c86eae3d4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:33.633880 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bd8f0-2995-4be8-affa-af6360317703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:33.635059 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bd8f0-2995-4be8-affa-af6360317703 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:33.635384 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bd8f0-2995-4be8-affa-af6360317703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:33.635562 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bd8f0-2995-4be8-affa-af6360317703 admin admin] PATH_INFO: `/v3/projects/ddd22b027e204f30848e177ea4b8dfe6/users/a2fc697f771d43b6a0f29f5adf8cb4d9/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:33.648911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0bd8f0-2995-4be8-affa-af6360317703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddd22b027e204f30848e177ea4b8dfe6, user_id=a2fc697f771d43b6a0f29f5adf8cb4d9, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:33.655708 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0bd8f0-2995-4be8-affa-af6360317703 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:33.660230 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1160/2340] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:31:33 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:31:33.665484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b618b82-121b-4b60-b68b-8feb361d0d4d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:33.675340 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1181/2341] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:31:33 2022] PUT /identity/v3/projects/ddd22b027e204f30848e177ea4b8dfe6/users/a2fc697f771d43b6a0f29f5adf8cb4d9/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:31:33.688675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b618b82-121b-4b60-b68b-8feb361d0d4d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:33.786079 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea06154-6602-499b-9d2d-a61966d7812c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:33.786884 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea06154-6602-499b-9d2d-a61966d7812c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:33.787248 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea06154-6602-499b-9d2d-a61966d7812c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:33.787604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea06154-6602-499b-9d2d-a61966d7812c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:33.788644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea06154-6602-499b-9d2d-a61966d7812c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:33.790900 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea06154-6602-499b-9d2d-a61966d7812c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:33.803031 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1161/2342] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:31:33.806749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bd8f0-2995-4be8-affa-af6360317703 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:33.820135 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bd8f0-2995-4be8-affa-af6360317703 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:33.826905 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a456ce5-d5bd-49b7-bc3a-eeac134b1846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:33.827607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a456ce5-d5bd-49b7-bc3a-eeac134b1846 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:33.827994 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a456ce5-d5bd-49b7-bc3a-eeac134b1846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:33.828282 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a456ce5-d5bd-49b7-bc3a-eeac134b1846 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:33.829903 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a456ce5-d5bd-49b7-bc3a-eeac134b1846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:33.831585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a456ce5-d5bd-49b7-bc3a-eeac134b1846 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:33.846678 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1182/2343] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:31:33.851082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea06154-6602-499b-9d2d-a61966d7812c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:33.863749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea06154-6602-499b-9d2d-a61966d7812c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:33.870399 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d96fc3-0a96-4cd4-b215-8545b1df86f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:33.871385 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d96fc3-0a96-4cd4-b215-8545b1df86f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:33.871670 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d96fc3-0a96-4cd4-b215-8545b1df86f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:33.872273 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d96fc3-0a96-4cd4-b215-8545b1df86f0 admin admin] PATH_INFO: `/v3/projects/0bd9fedf211d41878c431ee91f87aaf8/users/37a5ceff981447c4a65c0f2cec7607a0/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:33.890478 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d96fc3-0a96-4cd4-b215-8545b1df86f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd9fedf211d41878c431ee91f87aaf8, user_id=37a5ceff981447c4a65c0f2cec7607a0, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:33.893832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d96fc3-0a96-4cd4-b215-8545b1df86f0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:33.912268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1162/2344] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:31:33 2022] PUT /identity/v3/projects/0bd9fedf211d41878c431ee91f87aaf8/users/37a5ceff981447c4a65c0f2cec7607a0/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:31:33.929152 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6080e2f-5183-4bff-a70c-45ddcf868e9a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:33.929152 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6080e2f-5183-4bff-a70c-45ddcf868e9a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:33.929152 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6080e2f-5183-4bff-a70c-45ddcf868e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:33.971274 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c6080e2f-5183-4bff-a70c-45ddcf868e9a None None] MFA Rules not processed for user `37a5ceff981447c4a65c0f2cec7607a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:33.973729 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c6080e2f-5183-4bff-a70c-45ddcf868e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:34.073397 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1183/2345] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:31:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:31:34.086329 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d96fc3-0a96-4cd4-b215-8545b1df86f0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:34.088395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a7d96fc3-0a96-4cd4-b215-8545b1df86f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:34.165681 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d96fc3-0a96-4cd4-b215-8545b1df86f0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:34.170126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d086b8-e269-41a5-9d3b-50ce9ebc1890 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:34.170807 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d086b8-e269-41a5-9d3b-50ce9ebc1890 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:34.171098 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d086b8-e269-41a5-9d3b-50ce9ebc1890 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:34.171380 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d086b8-e269-41a5-9d3b-50ce9ebc1890 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:34.198445 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d086b8-e269-41a5-9d3b-50ce9ebc1890 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:34.200217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d086b8-e269-41a5-9d3b-50ce9ebc1890 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:34.232640 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1163/2346] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:31:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:31:35.877510 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fda96-5655-4c59-be6c-62bc9135a1cb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:35.878027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fda96-5655-4c59-be6c-62bc9135a1cb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:35.878110 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fda96-5655-4c59-be6c-62bc9135a1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:35.917609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-047fda96-5655-4c59-be6c-62bc9135a1cb None None] MFA Rules not processed for user `37a5ceff981447c4a65c0f2cec7607a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:35.918705 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-047fda96-5655-4c59-be6c-62bc9135a1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:35.942851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1184/2347] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:31:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:31:39.657409 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b478b51-7fae-4f6b-b63a-99e028feb420 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:39.657910 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b478b51-7fae-4f6b-b63a-99e028feb420 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:39.658013 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b478b51-7fae-4f6b-b63a-99e028feb420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:39.671640 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:31:39.711084 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0b478b51-7fae-4f6b-b63a-99e028feb420 None None] MFA Rules not processed for user `a2fc697f771d43b6a0f29f5adf8cb4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:39.712331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0b478b51-7fae-4f6b-b63a-99e028feb420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:39.820967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1164/2348] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:31:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:31:39.831477 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047fda96-5655-4c59-be6c-62bc9135a1cb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:39.838230 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-047fda96-5655-4c59-be6c-62bc9135a1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:39.940465 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:31:39.964445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047fda96-5655-4c59-be6c-62bc9135a1cb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:39.968517 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4a19d-1330-4e80-8940-a4b76ec55ee1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cee4d23797e430e9eeddc394ed7f50e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:39.969278 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4a19d-1330-4e80-8940-a4b76ec55ee1 service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:39.970143 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4a19d-1330-4e80-8940-a4b76ec55ee1 service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:39.970420 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4a19d-1330-4e80-8940-a4b76ec55ee1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:39.972392 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-17d4a19d-1330-4e80-8940-a4b76ec55ee1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:40.062673 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d4a19d-1330-4e80-8940-a4b76ec55ee1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:40.066622 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d4a19d-1330-4e80-8940-a4b76ec55ee1 service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:40.076707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1185/2349] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:31:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:31:40.242344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be039c09-b0c1-48b8-b214-b2ee5a365426 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:40.242741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be039c09-b0c1-48b8-b214-b2ee5a365426 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:40.243092 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be039c09-b0c1-48b8-b214-b2ee5a365426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:40.263966 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-be039c09-b0c1-48b8-b214-b2ee5a365426 None None] MFA Rules not processed for user `a09029144c454b099bea86aaa5229ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:40.265390 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-be039c09-b0c1-48b8-b214-b2ee5a365426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:40.337980 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1165/2350] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 18 10:31:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:31:40.350278 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4a19d-1330-4e80-8940-a4b76ec55ee1 service glance] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:40.351703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-17d4a19d-1330-4e80-8940-a4b76ec55ee1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:40.428899 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4a19d-1330-4e80-8940-a4b76ec55ee1 service glance] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:40.434451 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a935e-e1ae-452e-a458-b7e35f6c0152 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6168873f2846c8af73b903bba17eb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:40.435285 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a935e-e1ae-452e-a458-b7e35f6c0152 service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:40.435574 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a935e-e1ae-452e-a458-b7e35f6c0152 service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:40.435855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a935e-e1ae-452e-a458-b7e35f6c0152 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:40.446665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a935e-e1ae-452e-a458-b7e35f6c0152 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:40.448285 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a935e-e1ae-452e-a458-b7e35f6c0152 service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:40.460412 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1186/2351] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 18 10:31:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:31:47.761879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb42bf-75eb-492c-a1c4-35ea93483714 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:47.762515 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb42bf-75eb-492c-a1c4-35ea93483714 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:47.763356 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb42bf-75eb-492c-a1c4-35ea93483714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:47.796386 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b8cb42bf-75eb-492c-a1c4-35ea93483714 None None] MFA Rules not processed for user `8c9439cb644548b8b3b5fd5c610757b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:47.798831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b8cb42bf-75eb-492c-a1c4-35ea93483714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:47.888944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1166/2352] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:31:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:31:49.933343 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7468fa-648a-4a43-8894-db1817cec78b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:49.934510 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7468fa-648a-4a43-8894-db1817cec78b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:49.935043 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7468fa-648a-4a43-8894-db1817cec78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:49.970809 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-aa7468fa-648a-4a43-8894-db1817cec78b None None] MFA Rules not processed for user `1af30265de71409d876112350949a2a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:49.972618 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-aa7468fa-648a-4a43-8894-db1817cec78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:49.986544 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1187/2353] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:31:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:31:55.394533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc347572-72d5-4b2e-bd0d-c7e6d31cd391 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:55.395071 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc347572-72d5-4b2e-bd0d-c7e6d31cd391 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:55.395190 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc347572-72d5-4b2e-bd0d-c7e6d31cd391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:55.415343 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-dc347572-72d5-4b2e-bd0d-c7e6d31cd391 None None] MFA Rules not processed for user `a09029144c454b099bea86aaa5229ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:31:55.416814 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dc347572-72d5-4b2e-bd0d-c7e6d31cd391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:31:55.424548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1167/2354] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 18 10:31:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:31:55.440736 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7468fa-648a-4a43-8894-db1817cec78b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:31:55.454040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7468fa-648a-4a43-8894-db1817cec78b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:31:55.459441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615bdddb-0e6c-4564-8f15-c72d3c2e16bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6168873f2846c8af73b903bba17eb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:31:55.461009 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615bdddb-0e6c-4564-8f15-c72d3c2e16bb service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:31:55.461387 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615bdddb-0e6c-4564-8f15-c72d3c2e16bb service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:31:55.461784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615bdddb-0e6c-4564-8f15-c72d3c2e16bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:31:55.471954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615bdddb-0e6c-4564-8f15-c72d3c2e16bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:31:55.473628 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615bdddb-0e6c-4564-8f15-c72d3c2e16bb service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:31:55.483191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1188/2355] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 18 10:31:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:32:05.408614 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:32:05.479469 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=70 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:32:06.647139 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc347572-72d5-4b2e-bd0d-c7e6d31cd391 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:06.649074 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dc347572-72d5-4b2e-bd0d-c7e6d31cd391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:06.759918 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc347572-72d5-4b2e-bd0d-c7e6d31cd391 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:06.765944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ed99d-53ae-4f60-b96b-39950ca4835e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd07165f554a445169b2c3f0d173c0bb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:06.767194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ed99d-53ae-4f60-b96b-39950ca4835e service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:06.768759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ed99d-53ae-4f60-b96b-39950ca4835e service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:06.769236 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ed99d-53ae-4f60-b96b-39950ca4835e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:06.771465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5e4ed99d-53ae-4f60-b96b-39950ca4835e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:06.785465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ed99d-53ae-4f60-b96b-39950ca4835e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:06.787340 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ed99d-53ae-4f60-b96b-39950ca4835e service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:06.800012 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1168/2356] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:32:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:32:07.529917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a7ce7-6cb2-430b-b534-92c60601b372 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:07.530407 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a7ce7-6cb2-430b-b534-92c60601b372 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:07.530555 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a7ce7-6cb2-430b-b534-92c60601b372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:07.558211 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d26a7ce7-6cb2-430b-b534-92c60601b372 None None] MFA Rules not processed for user `a09029144c454b099bea86aaa5229ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:32:07.566945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d26a7ce7-6cb2-430b-b534-92c60601b372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:07.574561 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1189/2357] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 18 10:32:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:32:07.587237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ed99d-53ae-4f60-b96b-39950ca4835e service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:07.599214 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ed99d-53ae-4f60-b96b-39950ca4835e service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:07.604995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc4d7e-5165-43f2-a6e0-320c6e6c8133 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6168873f2846c8af73b903bba17eb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:07.605636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc4d7e-5165-43f2-a6e0-320c6e6c8133 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:07.605920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc4d7e-5165-43f2-a6e0-320c6e6c8133 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:07.606197 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc4d7e-5165-43f2-a6e0-320c6e6c8133 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:07.614647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcc4d7e-5165-43f2-a6e0-320c6e6c8133 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:07.616264 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcc4d7e-5165-43f2-a6e0-320c6e6c8133 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:07.624571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1169/2358] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 18 10:32:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:32:08.128677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdeb7b-ad64-47a0-9f90-29c2d612836e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:08.129047 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdeb7b-ad64-47a0-9f90-29c2d612836e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:08.129400 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdeb7b-ad64-47a0-9f90-29c2d612836e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:08.148027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1bcdeb7b-ad64-47a0-9f90-29c2d612836e None None] MFA Rules not processed for user `a09029144c454b099bea86aaa5229ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:32:08.149416 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1bcdeb7b-ad64-47a0-9f90-29c2d612836e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:08.162884 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1190/2359] 10.222.0.33 () {58 vars in 967 bytes} [Mon Jul 18 10:32:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:32:08.180821 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc4d7e-5165-43f2-a6e0-320c6e6c8133 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:08.191587 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc4d7e-5165-43f2-a6e0-320c6e6c8133 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:08.197692 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768bad9-e96c-42ed-96e7-68b6ae0d96a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6168873f2846c8af73b903bba17eb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:08.198351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768bad9-e96c-42ed-96e7-68b6ae0d96a6 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:08.198641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768bad9-e96c-42ed-96e7-68b6ae0d96a6 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:08.198923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768bad9-e96c-42ed-96e7-68b6ae0d96a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:08.209797 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768bad9-e96c-42ed-96e7-68b6ae0d96a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:08.211395 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768bad9-e96c-42ed-96e7-68b6ae0d96a6 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:08.220779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1170/2360] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 18 10:32:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:32:08.416495 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a33927-1fbd-4aee-8e62-55d1ea0360d7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:08.416778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a33927-1fbd-4aee-8e62-55d1ea0360d7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:08.417081 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a33927-1fbd-4aee-8e62-55d1ea0360d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:08.435371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-59a33927-1fbd-4aee-8e62-55d1ea0360d7 None None] MFA Rules not processed for user `a09029144c454b099bea86aaa5229ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:32:08.436605 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-59a33927-1fbd-4aee-8e62-55d1ea0360d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:08.444763 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1191/2361] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 18 10:32:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:32:08.457131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768bad9-e96c-42ed-96e7-68b6ae0d96a6 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:08.467322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768bad9-e96c-42ed-96e7-68b6ae0d96a6 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:08.472872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665aca3-b495-49c9-ac58-f996bcccc130 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6168873f2846c8af73b903bba17eb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:08.473565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665aca3-b495-49c9-ac58-f996bcccc130 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:08.473854 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665aca3-b495-49c9-ac58-f996bcccc130 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:08.474216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665aca3-b495-49c9-ac58-f996bcccc130 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:08.483718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665aca3-b495-49c9-ac58-f996bcccc130 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:08.486054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665aca3-b495-49c9-ac58-f996bcccc130 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:08.495007 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1171/2362] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 18 10:32:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:32:08.842316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d5b42-aff1-4985-8e26-1f5667afeca2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:08.842796 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d5b42-aff1-4985-8e26-1f5667afeca2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:08.843001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d5b42-aff1-4985-8e26-1f5667afeca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:08.862437 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d28d5b42-aff1-4985-8e26-1f5667afeca2 None None] MFA Rules not processed for user `a09029144c454b099bea86aaa5229ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:32:08.863804 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d28d5b42-aff1-4985-8e26-1f5667afeca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:08.871360 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1192/2363] 10.222.0.33 () {58 vars in 967 bytes} [Mon Jul 18 10:32:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:32:08.884970 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665aca3-b495-49c9-ac58-f996bcccc130 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:08.896009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665aca3-b495-49c9-ac58-f996bcccc130 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:08.900998 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1eec8d-9b41-41a5-9121-bf5128e82759 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6168873f2846c8af73b903bba17eb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:08.901669 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1eec8d-9b41-41a5-9121-bf5128e82759 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:08.901950 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1eec8d-9b41-41a5-9121-bf5128e82759 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:08.902268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1eec8d-9b41-41a5-9121-bf5128e82759 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:08.911034 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1eec8d-9b41-41a5-9121-bf5128e82759 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:08.912687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1eec8d-9b41-41a5-9121-bf5128e82759 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:08.927996 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1172/2364] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 18 10:32:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 10:32:11.182284 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d5b42-aff1-4985-8e26-1f5667afeca2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:11.245950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d5b42-aff1-4985-8e26-1f5667afeca2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:11.250549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc8b74-7f46-4a44-90c4-95128947cfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:11.252042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc8b74-7f46-4a44-90c4-95128947cfb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:11.252392 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc8b74-7f46-4a44-90c4-95128947cfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:11.252759 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc8b74-7f46-4a44-90c4-95128947cfb0 admin admin] PATH_INFO: `/v3/users/37a5ceff981447c4a65c0f2cec7607a0` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:11.254026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc8b74-7f46-4a44-90c4-95128947cfb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37a5ceff981447c4a65c0f2cec7607a0)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:11.256047 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc8b74-7f46-4a44-90c4-95128947cfb0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:11.305824 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8bcc8b74-7f46-4a44-90c4-95128947cfb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37a5ceff981447c4a65c0f2cec7607a0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:11.318492 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8bcc8b74-7f46-4a44-90c4-95128947cfb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37a5ceff981447c4a65c0f2cec7607a0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:11.319158 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8bcc8b74-7f46-4a44-90c4-95128947cfb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37a5ceff981447c4a65c0f2cec7607a0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:11.319756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8bcc8b74-7f46-4a44-90c4-95128947cfb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37a5ceff981447c4a65c0f2cec7607a0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:11.327013 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8bcc8b74-7f46-4a44-90c4-95128947cfb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37a5ceff981447c4a65c0f2cec7607a0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:11.339407 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1193/2365] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:32:11 2022] DELETE /identity/v3/users/37a5ceff981447c4a65c0f2cec7607a0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:32:11.343827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097a71b-9a1e-4419-a247-f68cd9378c49 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:11.344335 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097a71b-9a1e-4419-a247-f68cd9378c49 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:11.344435 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097a71b-9a1e-4419-a247-f68cd9378c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:11.362549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a097a71b-9a1e-4419-a247-f68cd9378c49 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:32:11.364064 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a097a71b-9a1e-4419-a247-f68cd9378c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:11.436533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1173/2366] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:32:11.446896 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc8b74-7f46-4a44-90c4-95128947cfb0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:11.448381 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8bcc8b74-7f46-4a44-90c4-95128947cfb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:11.507543 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc8b74-7f46-4a44-90c4-95128947cfb0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:11.511166 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db0e05-230b-49dc-b934-de9a26f07582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:11.511816 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db0e05-230b-49dc-b934-de9a26f07582 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:11.512180 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db0e05-230b-49dc-b934-de9a26f07582 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:11.512363 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db0e05-230b-49dc-b934-de9a26f07582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:11.519724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:32:11.521827 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db0e05-230b-49dc-b934-de9a26f07582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:11.523641 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db0e05-230b-49dc-b934-de9a26f07582 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:11.535636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1194/2367] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:32:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:32:11.703485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097a71b-9a1e-4419-a247-f68cd9378c49 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:11.704910 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a097a71b-9a1e-4419-a247-f68cd9378c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:11.716331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097a71b-9a1e-4419-a247-f68cd9378c49 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:11.721313 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ee410-a3af-4a6a-978d-406758f13a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:11.722166 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ee410-a3af-4a6a-978d-406758f13a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:11.722453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ee410-a3af-4a6a-978d-406758f13a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:11.722728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ee410-a3af-4a6a-978d-406758f13a99 admin admin] PATH_INFO: `/v3/projects/0bd9fedf211d41878c431ee91f87aaf8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:11.723898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ee410-a3af-4a6a-978d-406758f13a99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bd9fedf211d41878c431ee91f87aaf8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:11.725943 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ee410-a3af-4a6a-978d-406758f13a99 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:11.762319 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:32:11.774615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-370ee410-a3af-4a6a-978d-406758f13a99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bd9fedf211d41878c431ee91f87aaf8', 'request_id': 'req-370ee410-a3af-4a6a-978d-406758f13a99'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:11.779986 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-370ee410-a3af-4a6a-978d-406758f13a99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bd9fedf211d41878c431ee91f87aaf8', 'request_id': 'req-370ee410-a3af-4a6a-978d-406758f13a99'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:11.786743 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-370ee410-a3af-4a6a-978d-406758f13a99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bd9fedf211d41878c431ee91f87aaf8', 'request_id': 'req-370ee410-a3af-4a6a-978d-406758f13a99'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:11.799216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-370ee410-a3af-4a6a-978d-406758f13a99 admin admin] The token cache is being invalidate because project 0bd9fedf211d41878c431ee91f87aaf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:32:11.799525 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-370ee410-a3af-4a6a-978d-406758f13a99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:11.800134 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-370ee410-a3af-4a6a-978d-406758f13a99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:11.801665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1174/2368] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:32:11 2022] DELETE /identity/v3/projects/0bd9fedf211d41878c431ee91f87aaf8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:32:11.806606 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b943e-ab82-42c8-8912-6158b1e3f813 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:11.806898 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b943e-ab82-42c8-8912-6158b1e3f813 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:11.807178 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b943e-ab82-42c8-8912-6158b1e3f813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:11.824275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-924b943e-ab82-42c8-8912-6158b1e3f813 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:32:11.825516 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-924b943e-ab82-42c8-8912-6158b1e3f813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:11.889891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1195/2369] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:32:11.898094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af193fda-7a44-4c16-8214-4f3bedab66a3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:11.898384 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af193fda-7a44-4c16-8214-4f3bedab66a3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:11.898659 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af193fda-7a44-4c16-8214-4f3bedab66a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:11.917021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-af193fda-7a44-4c16-8214-4f3bedab66a3 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:32:11.918216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-af193fda-7a44-4c16-8214-4f3bedab66a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:11.925966 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1175/2370] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:32:11.930295 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b943e-ab82-42c8-8912-6158b1e3f813 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:11.939960 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b943e-ab82-42c8-8912-6158b1e3f813 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:11.944811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e06af-ed43-4cf8-9821-c511f7518617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:11.945461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e06af-ed43-4cf8-9821-c511f7518617 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:11.945749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e06af-ed43-4cf8-9821-c511f7518617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:11.946028 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e06af-ed43-4cf8-9821-c511f7518617 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:11.946810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e06af-ed43-4cf8-9821-c511f7518617 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:32:11.947066 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e06af-ed43-4cf8-9821-c511f7518617 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:11.948735 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e06af-ed43-4cf8-9821-c511f7518617 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:11.966822 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1196/2371] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:32:11.971794 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af193fda-7a44-4c16-8214-4f3bedab66a3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:11.981461 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af193fda-7a44-4c16-8214-4f3bedab66a3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:11.986183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3dc96-3b87-4c3e-b588-87ded121cf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:11.986826 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3dc96-3b87-4c3e-b588-87ded121cf8c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:11.987111 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3dc96-3b87-4c3e-b588-87ded121cf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:11.987386 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3dc96-3b87-4c3e-b588-87ded121cf8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:11.988082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3dc96-3b87-4c3e-b588-87ded121cf8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:11.990018 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3dc96-3b87-4c3e-b588-87ded121cf8c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:12.019027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1176/2372] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:32:11 2022] POST /identity/v3/projects => generated 397 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:32:12.023277 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e06af-ed43-4cf8-9821-c511f7518617 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:12.034213 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e06af-ed43-4cf8-9821-c511f7518617 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:12.093786 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9f18b-09de-41e9-9cdc-3f8f1c41d8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:12.095047 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9f18b-09de-41e9-9cdc-3f8f1c41d8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:12.095333 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9f18b-09de-41e9-9cdc-3f8f1c41d8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:12.095608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9f18b-09de-41e9-9cdc-3f8f1c41d8c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:12.096377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9f18b-09de-41e9-9cdc-3f8f1c41d8c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:12.098336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9f18b-09de-41e9-9cdc-3f8f1c41d8c3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:12.118546 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1197/2373] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:32:12 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:32:12.122516 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3dc96-3b87-4c3e-b588-87ded121cf8c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:12.132994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3dc96-3b87-4c3e-b588-87ded121cf8c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:12.137631 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f079a6-84c5-4f94-9d4c-b138ce17aa6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:12.138292 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f079a6-84c5-4f94-9d4c-b138ce17aa6f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:12.138580 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f079a6-84c5-4f94-9d4c-b138ce17aa6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:12.138861 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f079a6-84c5-4f94-9d4c-b138ce17aa6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:12.139587 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f079a6-84c5-4f94-9d4c-b138ce17aa6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:12.141870 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f079a6-84c5-4f94-9d4c-b138ce17aa6f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:12.152925 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1177/2374] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:32:12.157348 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9f18b-09de-41e9-9cdc-3f8f1c41d8c3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:12.167668 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9f18b-09de-41e9-9cdc-3f8f1c41d8c3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:12.172969 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef87aa4-b3fa-4e39-b8e5-4e41ffa2a759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:12.173773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef87aa4-b3fa-4e39-b8e5-4e41ffa2a759 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:12.174126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef87aa4-b3fa-4e39-b8e5-4e41ffa2a759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:12.174459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef87aa4-b3fa-4e39-b8e5-4e41ffa2a759 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:12.175290 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef87aa4-b3fa-4e39-b8e5-4e41ffa2a759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:12.177646 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef87aa4-b3fa-4e39-b8e5-4e41ffa2a759 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:12.190306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1198/2375] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:32:12.194534 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f079a6-84c5-4f94-9d4c-b138ce17aa6f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:12.205502 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f079a6-84c5-4f94-9d4c-b138ce17aa6f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:12.210412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764e2fc-7d20-4266-85ff-99f0b033a3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:12.211467 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764e2fc-7d20-4266-85ff-99f0b033a3c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:12.211876 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764e2fc-7d20-4266-85ff-99f0b033a3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:12.212237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764e2fc-7d20-4266-85ff-99f0b033a3c7 admin admin] PATH_INFO: `/v3/projects/ea5ab29f9bbf47fbbcdccff38b1c09e3/users/bd73dc9787824f5b84ea7efeb51c9361/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:12.227555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e764e2fc-7d20-4266-85ff-99f0b033a3c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea5ab29f9bbf47fbbcdccff38b1c09e3, user_id=bd73dc9787824f5b84ea7efeb51c9361, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:12.230743 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e764e2fc-7d20-4266-85ff-99f0b033a3c7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:12.245438 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1178/2376] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:32:12 2022] PUT /identity/v3/projects/ea5ab29f9bbf47fbbcdccff38b1c09e3/users/bd73dc9787824f5b84ea7efeb51c9361/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:32:12.256443 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcde10-630a-4bd4-bb6e-9421943a23dd None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:12.256801 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcde10-630a-4bd4-bb6e-9421943a23dd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:12.257087 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcde10-630a-4bd4-bb6e-9421943a23dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:12.283965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-68dcde10-630a-4bd4-bb6e-9421943a23dd None None] MFA Rules not processed for user `bd73dc9787824f5b84ea7efeb51c9361`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:32:12.285260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-68dcde10-630a-4bd4-bb6e-9421943a23dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:12.333118 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:32:12.350601 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1199/2377] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:32:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:32:12.362393 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764e2fc-7d20-4266-85ff-99f0b033a3c7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:12.364088 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e764e2fc-7d20-4266-85ff-99f0b033a3c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:12.443199 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764e2fc-7d20-4266-85ff-99f0b033a3c7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:12.446991 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65c3d2-f03b-46cc-9e16-06057d66c31c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:12.447644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65c3d2-f03b-46cc-9e16-06057d66c31c service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:12.448421 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65c3d2-f03b-46cc-9e16-06057d66c31c service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:12.448421 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65c3d2-f03b-46cc-9e16-06057d66c31c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:12.458296 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65c3d2-f03b-46cc-9e16-06057d66c31c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:12.460116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65c3d2-f03b-46cc-9e16-06057d66c31c service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:12.479728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1179/2378] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:32:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:32:20.600774 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcde10-630a-4bd4-bb6e-9421943a23dd None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:20.676806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcde10-630a-4bd4-bb6e-9421943a23dd None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:20.683674 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb57bcf-c15f-4b2c-87b2-c8fdc9292621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:20.685099 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb57bcf-c15f-4b2c-87b2-c8fdc9292621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:20.685679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb57bcf-c15f-4b2c-87b2-c8fdc9292621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:20.686107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb57bcf-c15f-4b2c-87b2-c8fdc9292621 admin admin] PATH_INFO: `/v3/users/bd73dc9787824f5b84ea7efeb51c9361` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:20.687660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb57bcf-c15f-4b2c-87b2-c8fdc9292621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd73dc9787824f5b84ea7efeb51c9361)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:20.689849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb57bcf-c15f-4b2c-87b2-c8fdc9292621 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:20.772600 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-acb57bcf-c15f-4b2c-87b2-c8fdc9292621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd73dc9787824f5b84ea7efeb51c9361'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:20.797489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-acb57bcf-c15f-4b2c-87b2-c8fdc9292621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd73dc9787824f5b84ea7efeb51c9361'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:20.799050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-acb57bcf-c15f-4b2c-87b2-c8fdc9292621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd73dc9787824f5b84ea7efeb51c9361'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:20.799657 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-acb57bcf-c15f-4b2c-87b2-c8fdc9292621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd73dc9787824f5b84ea7efeb51c9361'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:20.809861 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-acb57bcf-c15f-4b2c-87b2-c8fdc9292621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd73dc9787824f5b84ea7efeb51c9361'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:20.830661 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1200/2379] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:32:20 2022] DELETE /identity/v3/users/bd73dc9787824f5b84ea7efeb51c9361 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:32:20.835879 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030007b5-3448-4055-a4cd-001fc244e634 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:20.836378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030007b5-3448-4055-a4cd-001fc244e634 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:20.836696 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030007b5-3448-4055-a4cd-001fc244e634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:20.857774 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-030007b5-3448-4055-a4cd-001fc244e634 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:32:20.859212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-030007b5-3448-4055-a4cd-001fc244e634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:20.928324 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:32:20.944725 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1180/2380] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:32:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:32:20.952972 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb57bcf-c15f-4b2c-87b2-c8fdc9292621 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:20.954991 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-acb57bcf-c15f-4b2c-87b2-c8fdc9292621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:21.023996 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb57bcf-c15f-4b2c-87b2-c8fdc9292621 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:21.033950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9616d6d-1aa6-438b-887c-cfcf66a231b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:21.033950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9616d6d-1aa6-438b-887c-cfcf66a231b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:21.033950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9616d6d-1aa6-438b-887c-cfcf66a231b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:21.033950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9616d6d-1aa6-438b-887c-cfcf66a231b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:21.042192 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9616d6d-1aa6-438b-887c-cfcf66a231b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:21.044587 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9616d6d-1aa6-438b-887c-cfcf66a231b6 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:21.059686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1201/2381] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:32:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:32:21.386726 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030007b5-3448-4055-a4cd-001fc244e634 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:21.390335 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-030007b5-3448-4055-a4cd-001fc244e634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:21.448237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030007b5-3448-4055-a4cd-001fc244e634 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:21.454700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af5277-c335-4ab8-ae44-a23f985ab7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:21.455735 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af5277-c335-4ab8-ae44-a23f985ab7b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:21.456034 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af5277-c335-4ab8-ae44-a23f985ab7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:21.456314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af5277-c335-4ab8-ae44-a23f985ab7b4 admin admin] PATH_INFO: `/v3/projects/ea5ab29f9bbf47fbbcdccff38b1c09e3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:21.457635 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af5277-c335-4ab8-ae44-a23f985ab7b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea5ab29f9bbf47fbbcdccff38b1c09e3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:21.461692 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af5277-c335-4ab8-ae44-a23f985ab7b4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:21.529443 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c4af5277-c335-4ab8-ae44-a23f985ab7b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea5ab29f9bbf47fbbcdccff38b1c09e3', 'request_id': 'req-c4af5277-c335-4ab8-ae44-a23f985ab7b4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:21.540165 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c4af5277-c335-4ab8-ae44-a23f985ab7b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea5ab29f9bbf47fbbcdccff38b1c09e3', 'request_id': 'req-c4af5277-c335-4ab8-ae44-a23f985ab7b4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:21.549631 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c4af5277-c335-4ab8-ae44-a23f985ab7b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea5ab29f9bbf47fbbcdccff38b1c09e3', 'request_id': 'req-c4af5277-c335-4ab8-ae44-a23f985ab7b4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:21.566683 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c4af5277-c335-4ab8-ae44-a23f985ab7b4 admin admin] The token cache is being invalidate because project ea5ab29f9bbf47fbbcdccff38b1c09e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:32:21.567244 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c4af5277-c335-4ab8-ae44-a23f985ab7b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:21.567887 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c4af5277-c335-4ab8-ae44-a23f985ab7b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:32:21.572560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1181/2382] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:32:21 2022] DELETE /identity/v3/projects/ea5ab29f9bbf47fbbcdccff38b1c09e3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:32:21.575154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d78b16-6013-4385-8a87-e6d1d6842a01 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:21.575446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d78b16-6013-4385-8a87-e6d1d6842a01 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:21.575726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d78b16-6013-4385-8a87-e6d1d6842a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:21.594597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f9d78b16-6013-4385-8a87-e6d1d6842a01 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:32:21.595974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f9d78b16-6013-4385-8a87-e6d1d6842a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:21.607463 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:32:21.693394 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1202/2383] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:32:21.761082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bff9c-f135-423e-8ec3-0f8c241c2128 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:21.761373 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bff9c-f135-423e-8ec3-0f8c241c2128 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:21.761738 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bff9c-f135-423e-8ec3-0f8c241c2128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:21.782493 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b88bff9c-f135-423e-8ec3-0f8c241c2128 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:32:21.783719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b88bff9c-f135-423e-8ec3-0f8c241c2128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:21.795342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1182/2384] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:32:21.796086 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d78b16-6013-4385-8a87-e6d1d6842a01 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:21.807889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d78b16-6013-4385-8a87-e6d1d6842a01 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:21.814575 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b3c01-934a-41f4-9ed3-65206bd35615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:21.815266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b3c01-934a-41f4-9ed3-65206bd35615 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:21.815556 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b3c01-934a-41f4-9ed3-65206bd35615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:21.815840 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b3c01-934a-41f4-9ed3-65206bd35615 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:21.816613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b3c01-934a-41f4-9ed3-65206bd35615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:32:21.816976 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b3c01-934a-41f4-9ed3-65206bd35615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:21.819513 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b3c01-934a-41f4-9ed3-65206bd35615 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:21.833512 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1203/2385] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:32:21.837714 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bff9c-f135-423e-8ec3-0f8c241c2128 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:21.847372 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bff9c-f135-423e-8ec3-0f8c241c2128 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:21.852634 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be009c-1e36-49bf-be23-cf46f1d08b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:21.853582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be009c-1e36-49bf-be23-cf46f1d08b28 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:21.853867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be009c-1e36-49bf-be23-cf46f1d08b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:21.854146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be009c-1e36-49bf-be23-cf46f1d08b28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:21.854871 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be009c-1e36-49bf-be23-cf46f1d08b28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:21.857084 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be009c-1e36-49bf-be23-cf46f1d08b28 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:21.888861 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1183/2386] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:32:21 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:32:21.893539 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b3c01-934a-41f4-9ed3-65206bd35615 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:21.903713 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b3c01-934a-41f4-9ed3-65206bd35615 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:21.968774 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac120c4-890b-4a40-ad22-56ccb5c12708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:21.969491 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac120c4-890b-4a40-ad22-56ccb5c12708 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:21.969779 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac120c4-890b-4a40-ad22-56ccb5c12708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:21.970065 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac120c4-890b-4a40-ad22-56ccb5c12708 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:21.970764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac120c4-890b-4a40-ad22-56ccb5c12708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:21.972737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac120c4-890b-4a40-ad22-56ccb5c12708 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:21.997931 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1204/2387] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:32:21 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:32:22.002601 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be009c-1e36-49bf-be23-cf46f1d08b28 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:22.013650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be009c-1e36-49bf-be23-cf46f1d08b28 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:22.019717 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8c816-a024-4fe0-b7d1-94b46fb2ae8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:22.020399 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8c816-a024-4fe0-b7d1-94b46fb2ae8e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:22.020724 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8c816-a024-4fe0-b7d1-94b46fb2ae8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:22.021007 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8c816-a024-4fe0-b7d1-94b46fb2ae8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:22.021741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a8c816-a024-4fe0-b7d1-94b46fb2ae8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:22.023366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a8c816-a024-4fe0-b7d1-94b46fb2ae8e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:22.034295 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1184/2388] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:32:22.038358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac120c4-890b-4a40-ad22-56ccb5c12708 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:22.048809 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac120c4-890b-4a40-ad22-56ccb5c12708 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:22.054166 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4964e-3345-4592-b25d-5063d1491e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:22.054849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4964e-3345-4592-b25d-5063d1491e21 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:22.055140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4964e-3345-4592-b25d-5063d1491e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:22.055525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4964e-3345-4592-b25d-5063d1491e21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:22.056172 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4964e-3345-4592-b25d-5063d1491e21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:22.057911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4964e-3345-4592-b25d-5063d1491e21 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:22.069454 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1205/2389] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:32:22.073670 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8c816-a024-4fe0-b7d1-94b46fb2ae8e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:22.086720 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8c816-a024-4fe0-b7d1-94b46fb2ae8e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:22.096541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763259d-e9cf-43b5-a082-829a2e8f1ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:22.098680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763259d-e9cf-43b5-a082-829a2e8f1ef6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:22.099570 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763259d-e9cf-43b5-a082-829a2e8f1ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:22.099921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763259d-e9cf-43b5-a082-829a2e8f1ef6 admin admin] PATH_INFO: `/v3/projects/2c656b71bbd447cba7d11335c928dfda/users/4e9eb8d188e040bca894e038955f2402/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:22.140176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763259d-e9cf-43b5-a082-829a2e8f1ef6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c656b71bbd447cba7d11335c928dfda, user_id=4e9eb8d188e040bca894e038955f2402, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:22.143809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763259d-e9cf-43b5-a082-829a2e8f1ef6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:22.180458 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1185/2390] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:32:22 2022] PUT /identity/v3/projects/2c656b71bbd447cba7d11335c928dfda/users/4e9eb8d188e040bca894e038955f2402/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:32:22.185405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda0ca7-27a6-4b1b-83fd-d810fdaeb9d7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:22.185886 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda0ca7-27a6-4b1b-83fd-d810fdaeb9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:22.186366 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda0ca7-27a6-4b1b-83fd-d810fdaeb9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:22.206296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2cda0ca7-27a6-4b1b-83fd-d810fdaeb9d7 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:32:22.207628 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2cda0ca7-27a6-4b1b-83fd-d810fdaeb9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:22.275001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1206/2391] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:32:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:32:22.284712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763259d-e9cf-43b5-a082-829a2e8f1ef6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:22.286695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4763259d-e9cf-43b5-a082-829a2e8f1ef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:22.351963 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763259d-e9cf-43b5-a082-829a2e8f1ef6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:22.356874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4452e1-d2ca-4eb1-9334-2fe974444394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:22.357635 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4452e1-d2ca-4eb1-9334-2fe974444394 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:22.357994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4452e1-d2ca-4eb1-9334-2fe974444394 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:22.358484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4452e1-d2ca-4eb1-9334-2fe974444394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:22.382120 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4452e1-d2ca-4eb1-9334-2fe974444394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:22.383947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4452e1-d2ca-4eb1-9334-2fe974444394 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:22.395861 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1186/2392] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:32:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:32:27.309864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261dfbe-0334-4f50-a4d1-c3eb9a4f2ef9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:27.310380 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261dfbe-0334-4f50-a4d1-c3eb9a4f2ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:27.310461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261dfbe-0334-4f50-a4d1-c3eb9a4f2ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:27.339365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1261dfbe-0334-4f50-a4d1-c3eb9a4f2ef9 None None] MFA Rules not processed for user `4e9eb8d188e040bca894e038955f2402`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:32:27.340883 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1261dfbe-0334-4f50-a4d1-c3eb9a4f2ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:27.371564 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:32:27.434184 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1207/2393] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:32:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:32:27.438762 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4452e1-d2ca-4eb1-9334-2fe974444394 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:27.448629 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4452e1-d2ca-4eb1-9334-2fe974444394 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:27.453406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd75e29-05e6-42d4-acfa-bec8bffa564c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:27.454161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd75e29-05e6-42d4-acfa-bec8bffa564c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:27.454441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd75e29-05e6-42d4-acfa-bec8bffa564c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:27.454718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd75e29-05e6-42d4-acfa-bec8bffa564c admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:27.455454 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd75e29-05e6-42d4-acfa-bec8bffa564c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:27.457404 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd75e29-05e6-42d4-acfa-bec8bffa564c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:27.481216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:32:27.489615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1187/2394] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:32:27 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:32:27.496012 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261dfbe-0334-4f50-a4d1-c3eb9a4f2ef9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:27.511508 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261dfbe-0334-4f50-a4d1-c3eb9a4f2ef9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:27.580333 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafa5cc-5494-4b85-9c69-9843e65679df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:27.581047 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafa5cc-5494-4b85-9c69-9843e65679df admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:27.581329 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafa5cc-5494-4b85-9c69-9843e65679df admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:27.581606 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafa5cc-5494-4b85-9c69-9843e65679df admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:27.582302 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafa5cc-5494-4b85-9c69-9843e65679df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:27.584238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafa5cc-5494-4b85-9c69-9843e65679df admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:27.606556 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1208/2395] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:32:27 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:32:27.611161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd75e29-05e6-42d4-acfa-bec8bffa564c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:27.622334 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd75e29-05e6-42d4-acfa-bec8bffa564c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:27.638138 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d4d4d-95ab-438d-8e24-64eb03670fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:27.638835 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d4d4d-95ab-438d-8e24-64eb03670fbd admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:27.639118 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d4d4d-95ab-438d-8e24-64eb03670fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:27.639397 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d4d4d-95ab-438d-8e24-64eb03670fbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:27.640140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d4d4d-95ab-438d-8e24-64eb03670fbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:27.642336 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d4d4d-95ab-438d-8e24-64eb03670fbd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:27.654185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1188/2396] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:32:27.658567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafa5cc-5494-4b85-9c69-9843e65679df admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:27.669460 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafa5cc-5494-4b85-9c69-9843e65679df admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:27.674706 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f2e1c-dcc3-46cd-ab54-159646918b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:27.675685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f2e1c-dcc3-46cd-ab54-159646918b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:27.676181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f2e1c-dcc3-46cd-ab54-159646918b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:27.676537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f2e1c-dcc3-46cd-ab54-159646918b7f admin admin] PATH_INFO: `/v3/projects/e6b6505e83184b2f8d54ad2ea4a1b9c4/users/cd1c16a2086446c7b8cacd4a81f9af71/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:27.693179 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f2e1c-dcc3-46cd-ab54-159646918b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6b6505e83184b2f8d54ad2ea4a1b9c4, user_id=cd1c16a2086446c7b8cacd4a81f9af71, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:27.695866 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f2e1c-dcc3-46cd-ab54-159646918b7f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:27.710533 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1209/2397] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:32:27 2022] PUT /identity/v3/projects/e6b6505e83184b2f8d54ad2ea4a1b9c4/users/cd1c16a2086446c7b8cacd4a81f9af71/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:32:27.714485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d4d4d-95ab-438d-8e24-64eb03670fbd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:27.725773 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d4d4d-95ab-438d-8e24-64eb03670fbd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:27.786018 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4099167-4d2c-49bb-a27f-4e3b347ee4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:27.786701 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4099167-4d2c-49bb-a27f-4e3b347ee4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:27.786992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4099167-4d2c-49bb-a27f-4e3b347ee4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:27.787364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4099167-4d2c-49bb-a27f-4e3b347ee4e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:27.788051 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4099167-4d2c-49bb-a27f-4e3b347ee4e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:27.793102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4099167-4d2c-49bb-a27f-4e3b347ee4e3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:27.805652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1189/2398] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:32:27.809632 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f2e1c-dcc3-46cd-ab54-159646918b7f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:27.830525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f2e1c-dcc3-46cd-ab54-159646918b7f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:27.835395 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df38435-cfc4-4498-908a-e515346057a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:27.836286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df38435-cfc4-4498-908a-e515346057a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:27.836663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df38435-cfc4-4498-908a-e515346057a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:27.836943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df38435-cfc4-4498-908a-e515346057a8 admin admin] PATH_INFO: `/v3/projects/e6b6505e83184b2f8d54ad2ea4a1b9c4/users/cd1c16a2086446c7b8cacd4a81f9af71/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:27.838911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df38435-cfc4-4498-908a-e515346057a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6b6505e83184b2f8d54ad2ea4a1b9c4, user_id=cd1c16a2086446c7b8cacd4a81f9af71, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:27.841602 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df38435-cfc4-4498-908a-e515346057a8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:27.856297 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1210/2399] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:32:27 2022] PUT /identity/v3/projects/e6b6505e83184b2f8d54ad2ea4a1b9c4/users/cd1c16a2086446c7b8cacd4a81f9af71/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:32:32.712934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca00fae-e0e2-4cf4-b444-9f40a263a85d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:32.713431 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca00fae-e0e2-4cf4-b444-9f40a263a85d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:32.713522 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca00fae-e0e2-4cf4-b444-9f40a263a85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:32.741296 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-dca00fae-e0e2-4cf4-b444-9f40a263a85d None None] MFA Rules not processed for user `cd1c16a2086446c7b8cacd4a81f9af71`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:32:32.742523 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dca00fae-e0e2-4cf4-b444-9f40a263a85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:32.836965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1190/2400] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:32:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:32:32.846232 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df38435-cfc4-4498-908a-e515346057a8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:32.848004 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8df38435-cfc4-4498-908a-e515346057a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:32.918250 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:32:32.934925 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df38435-cfc4-4498-908a-e515346057a8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:32.940565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf936787-bc12-48e3-831a-b142ad3c90e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:32.941218 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf936787-bc12-48e3-831a-b142ad3c90e7 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:32.941506 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf936787-bc12-48e3-831a-b142ad3c90e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:32.941776 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf936787-bc12-48e3-831a-b142ad3c90e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:33.003044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf936787-bc12-48e3-831a-b142ad3c90e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:33.004800 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf936787-bc12-48e3-831a-b142ad3c90e7 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:33.014191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1211/2401] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:32:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:32:37.690787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca00fae-e0e2-4cf4-b444-9f40a263a85d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:32:37.692182 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dca00fae-e0e2-4cf4-b444-9f40a263a85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:37.751665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca00fae-e0e2-4cf4-b444-9f40a263a85d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:32:37.756081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575190c0-2800-4d8e-ad10-643eaf750a7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:32:37.757455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575190c0-2800-4d8e-ad10-643eaf750a7f service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:37.757455 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575190c0-2800-4d8e-ad10-643eaf750a7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:37.757791 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575190c0-2800-4d8e-ad10-643eaf750a7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:37.759717 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-575190c0-2800-4d8e-ad10-643eaf750a7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:37.786912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:32:37.830515 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575190c0-2800-4d8e-ad10-643eaf750a7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:32:37.831967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575190c0-2800-4d8e-ad10-643eaf750a7f service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:32:37.841750 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1191/2402] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:32:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:32:43.009910 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=28 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:32:47.837561 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=18 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:32:48.327220 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a3520-a09d-4f7c-9a7d-f43b8044d203 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:48.327728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a3520-a09d-4f7c-9a7d-f43b8044d203 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:48.327827 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a3520-a09d-4f7c-9a7d-f43b8044d203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:48.345714 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-763a3520-a09d-4f7c-9a7d-f43b8044d203 None None] MFA Rules not processed for user `f25c72896a2d4d619e5ad4b5a0a79aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:32:48.346911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-763a3520-a09d-4f7c-9a7d-f43b8044d203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:48.399935 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1212/2403] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:32:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:32:52.790280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64995f42-867e-4d5e-9290-5dbea4a1eac4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:52.790851 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64995f42-867e-4d5e-9290-5dbea4a1eac4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:52.790851 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64995f42-867e-4d5e-9290-5dbea4a1eac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:52.808650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-64995f42-867e-4d5e-9290-5dbea4a1eac4 None None] MFA Rules not processed for user `4e9eb8d188e040bca894e038955f2402`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:32:52.809875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-64995f42-867e-4d5e-9290-5dbea4a1eac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:52.817471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1192/2404] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:32:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:32:55.566214 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815da902-070d-40cf-ae40-5fffcfaa17fd None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:32:55.567117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815da902-070d-40cf-ae40-5fffcfaa17fd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:32:55.567555 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815da902-070d-40cf-ae40-5fffcfaa17fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:32:55.587140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-815da902-070d-40cf-ae40-5fffcfaa17fd None None] MFA Rules not processed for user `72a431c9708f4694bc0d514133e76088`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:32:55.588638 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-815da902-070d-40cf-ae40-5fffcfaa17fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:32:55.655113 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1213/2405] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:32:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:33:02.803363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:33:05.643516 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=50 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:33:10.099842 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64995f42-867e-4d5e-9290-5dbea4a1eac4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:33:10.102308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-64995f42-867e-4d5e-9290-5dbea4a1eac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:33:10.177005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64995f42-867e-4d5e-9290-5dbea4a1eac4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:33:10.183981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243ca70-c6d0-4b98-a44f-62aa43cf1500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:33:10.184961 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243ca70-c6d0-4b98-a44f-62aa43cf1500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:33:10.185247 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243ca70-c6d0-4b98-a44f-62aa43cf1500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:33:10.185520 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243ca70-c6d0-4b98-a44f-62aa43cf1500 admin admin] PATH_INFO: `/v3/users/8c9439cb644548b8b3b5fd5c610757b9` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:33:10.187260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243ca70-c6d0-4b98-a44f-62aa43cf1500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c9439cb644548b8b3b5fd5c610757b9)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:33:10.189233 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243ca70-c6d0-4b98-a44f-62aa43cf1500 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:33:10.244848 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7243ca70-c6d0-4b98-a44f-62aa43cf1500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c9439cb644548b8b3b5fd5c610757b9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:33:10.245501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7243ca70-c6d0-4b98-a44f-62aa43cf1500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c9439cb644548b8b3b5fd5c610757b9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:33:10.246168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7243ca70-c6d0-4b98-a44f-62aa43cf1500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c9439cb644548b8b3b5fd5c610757b9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:33:10.254285 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7243ca70-c6d0-4b98-a44f-62aa43cf1500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c9439cb644548b8b3b5fd5c610757b9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:33:10.267498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7243ca70-c6d0-4b98-a44f-62aa43cf1500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c9439cb644548b8b3b5fd5c610757b9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:33:10.283882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1193/2406] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:33:10 2022] DELETE /identity/v3/users/8c9439cb644548b8b3b5fd5c610757b9 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:33:10.469365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815da902-070d-40cf-ae40-5fffcfaa17fd None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:33:10.481370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815da902-070d-40cf-ae40-5fffcfaa17fd None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:33:10.551738 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b33b7-1706-403c-9d7d-eb2fbcdaec0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:33:10.552693 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b33b7-1706-403c-9d7d-eb2fbcdaec0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:33:10.552985 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b33b7-1706-403c-9d7d-eb2fbcdaec0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:33:10.553265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b33b7-1706-403c-9d7d-eb2fbcdaec0f admin admin] PATH_INFO: `/v3/projects/ee6203312784435b8cd1fdfede3fb597` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:33:10.554486 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b33b7-1706-403c-9d7d-eb2fbcdaec0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee6203312784435b8cd1fdfede3fb597)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:33:10.556464 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b33b7-1706-403c-9d7d-eb2fbcdaec0f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:33:10.603432 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0f1b33b7-1706-403c-9d7d-eb2fbcdaec0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee6203312784435b8cd1fdfede3fb597', 'request_id': 'req-0f1b33b7-1706-403c-9d7d-eb2fbcdaec0f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:33:10.609459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0f1b33b7-1706-403c-9d7d-eb2fbcdaec0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee6203312784435b8cd1fdfede3fb597', 'request_id': 'req-0f1b33b7-1706-403c-9d7d-eb2fbcdaec0f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:33:10.615639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0f1b33b7-1706-403c-9d7d-eb2fbcdaec0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee6203312784435b8cd1fdfede3fb597', 'request_id': 'req-0f1b33b7-1706-403c-9d7d-eb2fbcdaec0f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:33:10.630704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0f1b33b7-1706-403c-9d7d-eb2fbcdaec0f admin admin] The token cache is being invalidate because project ee6203312784435b8cd1fdfede3fb597 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:33:10.631000 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0f1b33b7-1706-403c-9d7d-eb2fbcdaec0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:33:10.631623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0f1b33b7-1706-403c-9d7d-eb2fbcdaec0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:33:10.633377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1214/2407] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:33:10 2022] DELETE /identity/v3/projects/ee6203312784435b8cd1fdfede3fb597 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:33:16.601996 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243ca70-c6d0-4b98-a44f-62aa43cf1500 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:33:16.603695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7243ca70-c6d0-4b98-a44f-62aa43cf1500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:33:16.658652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243ca70-c6d0-4b98-a44f-62aa43cf1500 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:33:16.689381 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:33:16.708136 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdc15d-8dda-4f21-9106-277fce69ac3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:33:16.708818 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdc15d-8dda-4f21-9106-277fce69ac3d service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:33:16.709101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdc15d-8dda-4f21-9106-277fce69ac3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:33:16.709378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdc15d-8dda-4f21-9106-277fce69ac3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:33:16.711173 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cabdc15d-8dda-4f21-9106-277fce69ac3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:33:16.773422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdc15d-8dda-4f21-9106-277fce69ac3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:33:16.774890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdc15d-8dda-4f21-9106-277fce69ac3d service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:33:16.784637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1194/2408] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:33:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:33:20.627454 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:33:26.780242 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=34 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:33:49.983048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b1973-3038-4508-a56f-2c2b03102e57 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:33:49.983592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b1973-3038-4508-a56f-2c2b03102e57 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:33:49.983592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b1973-3038-4508-a56f-2c2b03102e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:33:50.001823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-918b1973-3038-4508-a56f-2c2b03102e57 None None] MFA Rules not processed for user `986f90dfbad14324b3fffbd790d60a37`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:33:50.003095 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-918b1973-3038-4508-a56f-2c2b03102e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:33:50.058685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1215/2409] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:33:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:33:53.145204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdc15d-8dda-4f21-9106-277fce69ac3d service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:33:53.147129 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cabdc15d-8dda-4f21-9106-277fce69ac3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:33:53.162140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdc15d-8dda-4f21-9106-277fce69ac3d service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:33:53.169254 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b479d-f7e6-4939-914b-507818abf03a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:33:53.169944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b479d-f7e6-4939-914b-507818abf03a service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:33:53.170234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b479d-f7e6-4939-914b-507818abf03a service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:33:53.170517 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b479d-f7e6-4939-914b-507818abf03a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:33:53.173810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f06b479d-f7e6-4939-914b-507818abf03a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:33:53.318041 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06b479d-f7e6-4939-914b-507818abf03a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:33:53.319838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06b479d-f7e6-4939-914b-507818abf03a service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:33:53.336572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1195/2410] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:33:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:33:55.782294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b1973-3038-4508-a56f-2c2b03102e57 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:33:55.792076 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b1973-3038-4508-a56f-2c2b03102e57 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:33:55.795663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe182833-4c1c-410d-b0cb-fdf4479c58cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:33:55.796301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe182833-4c1c-410d-b0cb-fdf4479c58cf service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:33:55.796614 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe182833-4c1c-410d-b0cb-fdf4479c58cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:33:55.796893 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe182833-4c1c-410d-b0cb-fdf4479c58cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:33:55.798708 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fe182833-4c1c-410d-b0cb-fdf4479c58cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:33:55.812094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe182833-4c1c-410d-b0cb-fdf4479c58cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:33:55.814111 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe182833-4c1c-410d-b0cb-fdf4479c58cf service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:33:55.825315 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1216/2411] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:33:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:01.350320 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b479d-f7e6-4939-914b-507818abf03a service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:01.351983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f06b479d-f7e6-4939-914b-507818abf03a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:01.422235 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b479d-f7e6-4939-914b-507818abf03a service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:01.426763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e773a52-0ea2-4cc2-a651-b7ea703a6f48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:01.427548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e773a52-0ea2-4cc2-a651-b7ea703a6f48 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:01.427900 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e773a52-0ea2-4cc2-a651-b7ea703a6f48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:01.428247 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e773a52-0ea2-4cc2-a651-b7ea703a6f48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:01.430218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5e773a52-0ea2-4cc2-a651-b7ea703a6f48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:01.490009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e773a52-0ea2-4cc2-a651-b7ea703a6f48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:01.491450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e773a52-0ea2-4cc2-a651-b7ea703a6f48 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:01.501215 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1196/2412] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:04.945662 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe182833-4c1c-410d-b0cb-fdf4479c58cf service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:04.947090 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fe182833-4c1c-410d-b0cb-fdf4479c58cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:04.961278 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe182833-4c1c-410d-b0cb-fdf4479c58cf service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:04.966980 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6af6c-b9fa-4188-9a7f-a07b9ebc7f81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:04.967658 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6af6c-b9fa-4188-9a7f-a07b9ebc7f81 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:04.967970 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6af6c-b9fa-4188-9a7f-a07b9ebc7f81 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:04.968250 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6af6c-b9fa-4188-9a7f-a07b9ebc7f81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:04.970675 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dfd6af6c-b9fa-4188-9a7f-a07b9ebc7f81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:05.013221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e773a52-0ea2-4cc2-a651-b7ea703a6f48 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:05.023320 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e773a52-0ea2-4cc2-a651-b7ea703a6f48 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:05.027154 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e10e8d-843d-46ff-9db7-c6a690a5d22f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:05.027807 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e10e8d-843d-46ff-9db7-c6a690a5d22f service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:05.028115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e10e8d-843d-46ff-9db7-c6a690a5d22f service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:05.028350 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e10e8d-843d-46ff-9db7-c6a690a5d22f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:05.033393 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6af6c-b9fa-4188-9a7f-a07b9ebc7f81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:05.035023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6af6c-b9fa-4188-9a7f-a07b9ebc7f81 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:05.037774 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e10e8d-843d-46ff-9db7-c6a690a5d22f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:05.039307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e10e8d-843d-46ff-9db7-c6a690a5d22f service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:05.044624 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1217/2413] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:05.055406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1197/2414] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:06.414638 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6af6c-b9fa-4188-9a7f-a07b9ebc7f81 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:06.424550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6af6c-b9fa-4188-9a7f-a07b9ebc7f81 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:06.428142 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd12ec-ebf1-4323-950d-2aa737efe9a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:06.428808 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd12ec-ebf1-4323-950d-2aa737efe9a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:06.429089 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd12ec-ebf1-4323-950d-2aa737efe9a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:06.429364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd12ec-ebf1-4323-950d-2aa737efe9a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:06.431120 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b5cd12ec-ebf1-4323-950d-2aa737efe9a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:06.494072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd12ec-ebf1-4323-950d-2aa737efe9a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:06.495505 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd12ec-ebf1-4323-950d-2aa737efe9a5 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:06.505592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1218/2415] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:09.739071 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e10e8d-843d-46ff-9db7-c6a690a5d22f service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:09.740604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c2e10e8d-843d-46ff-9db7-c6a690a5d22f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:09.755101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e10e8d-843d-46ff-9db7-c6a690a5d22f service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:09.759572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b33561-1fc2-4f28-a5e9-67e6f634d4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:09.760429 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b33561-1fc2-4f28-a5e9-67e6f634d4d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:09.760748 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b33561-1fc2-4f28-a5e9-67e6f634d4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:09.761025 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b33561-1fc2-4f28-a5e9-67e6f634d4d6 admin admin] PATH_INFO: `/v3/users/72a431c9708f4694bc0d514133e76088` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:09.762198 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b33561-1fc2-4f28-a5e9-67e6f634d4d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72a431c9708f4694bc0d514133e76088)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:09.764112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b33561-1fc2-4f28-a5e9-67e6f634d4d6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:09.786080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:09.810503 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-75b33561-1fc2-4f28-a5e9-67e6f634d4d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72a431c9708f4694bc0d514133e76088'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:09.811158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-75b33561-1fc2-4f28-a5e9-67e6f634d4d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72a431c9708f4694bc0d514133e76088'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:09.811759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-75b33561-1fc2-4f28-a5e9-67e6f634d4d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72a431c9708f4694bc0d514133e76088'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:09.823660 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-75b33561-1fc2-4f28-a5e9-67e6f634d4d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72a431c9708f4694bc0d514133e76088'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:09.836254 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-75b33561-1fc2-4f28-a5e9-67e6f634d4d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72a431c9708f4694bc0d514133e76088'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:09.849485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1198/2416] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:34:09 2022] DELETE /identity/v3/users/72a431c9708f4694bc0d514133e76088 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:34:10.025450 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd12ec-ebf1-4323-950d-2aa737efe9a5 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:10.045148 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd12ec-ebf1-4323-950d-2aa737efe9a5 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:10.074851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:10.116927 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be4a78-c5ff-4942-8477-1077518814aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:10.117793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be4a78-c5ff-4942-8477-1077518814aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:10.118083 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be4a78-c5ff-4942-8477-1077518814aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:10.118363 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be4a78-c5ff-4942-8477-1077518814aa admin admin] PATH_INFO: `/v3/projects/8a1921237622439bab89676e11477c13` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:10.119544 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be4a78-c5ff-4942-8477-1077518814aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a1921237622439bab89676e11477c13)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:10.121529 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be4a78-c5ff-4942-8477-1077518814aa admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:10.169532 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b2be4a78-c5ff-4942-8477-1077518814aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a1921237622439bab89676e11477c13', 'request_id': 'req-b2be4a78-c5ff-4942-8477-1077518814aa'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:10.176085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b2be4a78-c5ff-4942-8477-1077518814aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a1921237622439bab89676e11477c13', 'request_id': 'req-b2be4a78-c5ff-4942-8477-1077518814aa'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:10.184060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b2be4a78-c5ff-4942-8477-1077518814aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a1921237622439bab89676e11477c13', 'request_id': 'req-b2be4a78-c5ff-4942-8477-1077518814aa'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:10.196216 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b2be4a78-c5ff-4942-8477-1077518814aa admin admin] The token cache is being invalidate because project 8a1921237622439bab89676e11477c13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:34:10.196532 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b2be4a78-c5ff-4942-8477-1077518814aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:10.201930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b2be4a78-c5ff-4942-8477-1077518814aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:10.203985 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1219/2417] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:34:10 2022] DELETE /identity/v3/projects/8a1921237622439bab89676e11477c13 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:34:10.208875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a5ae0-48c4-49a0-8a20-d537f488913c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:10.209197 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a5ae0-48c4-49a0-8a20-d537f488913c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:10.209480 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a5ae0-48c4-49a0-8a20-d537f488913c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:10.228055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-941a5ae0-48c4-49a0-8a20-d537f488913c None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:10.229292 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-941a5ae0-48c4-49a0-8a20-d537f488913c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:10.287168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1199/2418] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:10.294753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50d885-8d73-4ac5-9536-baaa6c7e9967 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:10.295065 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50d885-8d73-4ac5-9536-baaa6c7e9967 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:10.295345 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50d885-8d73-4ac5-9536-baaa6c7e9967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:10.313325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5f50d885-8d73-4ac5-9536-baaa6c7e9967 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:10.314521 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5f50d885-8d73-4ac5-9536-baaa6c7e9967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:10.322311 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1220/2419] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:10.326413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a5ae0-48c4-49a0-8a20-d537f488913c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:10.336447 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a5ae0-48c4-49a0-8a20-d537f488913c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:10.340977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a6b77-a01c-4378-b9b3-216381d297ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:10.341628 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a6b77-a01c-4378-b9b3-216381d297ba admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:10.341913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a6b77-a01c-4378-b9b3-216381d297ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:10.342192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a6b77-a01c-4378-b9b3-216381d297ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:10.342913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a6b77-a01c-4378-b9b3-216381d297ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:34:10.343219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a6b77-a01c-4378-b9b3-216381d297ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:10.345391 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a6b77-a01c-4378-b9b3-216381d297ba admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:10.357063 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1200/2420] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:34:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:34:10.361420 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50d885-8d73-4ac5-9536-baaa6c7e9967 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:10.371205 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50d885-8d73-4ac5-9536-baaa6c7e9967 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:10.377080 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01faca59-b5f0-4713-a199-e86881652ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:10.378004 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01faca59-b5f0-4713-a199-e86881652ebb admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:10.378430 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01faca59-b5f0-4713-a199-e86881652ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:10.378738 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01faca59-b5f0-4713-a199-e86881652ebb admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:10.379456 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01faca59-b5f0-4713-a199-e86881652ebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:10.381427 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01faca59-b5f0-4713-a199-e86881652ebb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:10.410344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1221/2421] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:34:10 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:34:10.414392 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a6b77-a01c-4378-b9b3-216381d297ba admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:10.424271 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a6b77-a01c-4378-b9b3-216381d297ba admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:10.486552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd29fc4-ea20-4d36-9a26-41ba3416bfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:10.487237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd29fc4-ea20-4d36-9a26-41ba3416bfe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:10.487569 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd29fc4-ea20-4d36-9a26-41ba3416bfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:10.487850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd29fc4-ea20-4d36-9a26-41ba3416bfe5 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:10.488590 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd29fc4-ea20-4d36-9a26-41ba3416bfe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:10.490666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd29fc4-ea20-4d36-9a26-41ba3416bfe5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:10.514277 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1201/2422] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:34:10 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:34:10.519002 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01faca59-b5f0-4713-a199-e86881652ebb admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:10.530123 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01faca59-b5f0-4713-a199-e86881652ebb admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:10.535344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073052a-c502-4f48-8db6-5c3a2b4583a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:10.536006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073052a-c502-4f48-8db6-5c3a2b4583a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:10.536293 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073052a-c502-4f48-8db6-5c3a2b4583a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:10.536698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073052a-c502-4f48-8db6-5c3a2b4583a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:10.537504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073052a-c502-4f48-8db6-5c3a2b4583a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:10.539066 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073052a-c502-4f48-8db6-5c3a2b4583a3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:10.549476 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1222/2423] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:34:10.553553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd29fc4-ea20-4d36-9a26-41ba3416bfe5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:10.563505 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd29fc4-ea20-4d36-9a26-41ba3416bfe5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:10.568034 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bf41e-a34b-4fc5-acbe-62faac47012f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:10.568724 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bf41e-a34b-4fc5-acbe-62faac47012f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:10.569015 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bf41e-a34b-4fc5-acbe-62faac47012f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:10.569297 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bf41e-a34b-4fc5-acbe-62faac47012f admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:10.570024 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8bf41e-a34b-4fc5-acbe-62faac47012f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:10.571622 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8bf41e-a34b-4fc5-acbe-62faac47012f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:10.582004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1202/2424] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:34:10.586223 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073052a-c502-4f48-8db6-5c3a2b4583a3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:10.596890 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073052a-c502-4f48-8db6-5c3a2b4583a3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:10.601623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cd056-6983-4aaa-8b79-9764801d0f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:10.602777 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cd056-6983-4aaa-8b79-9764801d0f1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:10.603060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cd056-6983-4aaa-8b79-9764801d0f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:10.603334 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cd056-6983-4aaa-8b79-9764801d0f1c admin admin] PATH_INFO: `/v3/projects/feefb94984d54cbbaa70b963d956a520/users/cacc328f0f3540299310a0e53afe6acb/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:10.617376 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90cd056-6983-4aaa-8b79-9764801d0f1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=feefb94984d54cbbaa70b963d956a520, user_id=cacc328f0f3540299310a0e53afe6acb, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:10.620056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90cd056-6983-4aaa-8b79-9764801d0f1c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:10.633386 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1223/2425] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:34:10 2022] PUT /identity/v3/projects/feefb94984d54cbbaa70b963d956a520/users/cacc328f0f3540299310a0e53afe6acb/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:34:10.639057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27c63d-46a1-49a8-84b9-d0a00c54b196 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:10.639363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27c63d-46a1-49a8-84b9-d0a00c54b196 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:10.639649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27c63d-46a1-49a8-84b9-d0a00c54b196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:10.647003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:10.658925 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ed27c63d-46a1-49a8-84b9-d0a00c54b196 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:10.660545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ed27c63d-46a1-49a8-84b9-d0a00c54b196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:10.717662 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1203/2426] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:10.727604 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cd056-6983-4aaa-8b79-9764801d0f1c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:10.729409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f90cd056-6983-4aaa-8b79-9764801d0f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:10.785954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cd056-6983-4aaa-8b79-9764801d0f1c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:10.789726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0624c-27bf-4268-b2ed-4830a7a01364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:10.790317 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0624c-27bf-4268-b2ed-4830a7a01364 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:10.790605 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0624c-27bf-4268-b2ed-4830a7a01364 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:10.790885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0624c-27bf-4268-b2ed-4830a7a01364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:10.796924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:10.801326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0624c-27bf-4268-b2ed-4830a7a01364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:10.803082 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0624c-27bf-4268-b2ed-4830a7a01364 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:10.813782 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1224/2427] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:15.035601 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae1a92-ab3b-45e8-bc86-13d9d89432df None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:15.036059 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae1a92-ab3b-45e8-bc86-13d9d89432df None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:15.036250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae1a92-ab3b-45e8-bc86-13d9d89432df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:15.098483 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-15ae1a92-ab3b-45e8-bc86-13d9d89432df None None] MFA Rules not processed for user `cacc328f0f3540299310a0e53afe6acb`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:15.099714 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-15ae1a92-ab3b-45e8-bc86-13d9d89432df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:15.176722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1204/2428] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:34:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:15.193658 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0624c-27bf-4268-b2ed-4830a7a01364 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:15.195090 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-42a0624c-27bf-4268-b2ed-4830a7a01364 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:15.308965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0624c-27bf-4268-b2ed-4830a7a01364 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:15.333410 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e2ffe-19f6-4f4a-bdd1-deb9a82c30a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:15.334101 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e2ffe-19f6-4f4a-bdd1-deb9a82c30a6 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:15.334391 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e2ffe-19f6-4f4a-bdd1-deb9a82c30a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:15.334668 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e2ffe-19f6-4f4a-bdd1-deb9a82c30a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:15.347216 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e2ffe-19f6-4f4a-bdd1-deb9a82c30a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:15.349039 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e2ffe-19f6-4f4a-bdd1-deb9a82c30a6 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:15.361424 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1225/2429] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:17.900592 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49e1f0-192d-4965-b4a6-eabc3e43f09e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:17.901979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49e1f0-192d-4965-b4a6-eabc3e43f09e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:17.902256 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49e1f0-192d-4965-b4a6-eabc3e43f09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:17.927886 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fc49e1f0-192d-4965-b4a6-eabc3e43f09e None None] MFA Rules not processed for user `cacc328f0f3540299310a0e53afe6acb`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:17.929809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fc49e1f0-192d-4965-b4a6-eabc3e43f09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:17.939580 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1205/2430] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:34:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:18.410734 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e2ffe-19f6-4f4a-bdd1-deb9a82c30a6 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:18.427003 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e2ffe-19f6-4f4a-bdd1-deb9a82c30a6 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:18.434054 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606699f-4f4d-44df-8895-ba8ce19ffe18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:18.434724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606699f-4f4d-44df-8895-ba8ce19ffe18 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:18.435010 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606699f-4f4d-44df-8895-ba8ce19ffe18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:18.435291 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606699f-4f4d-44df-8895-ba8ce19ffe18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:18.438863 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e606699f-4f4d-44df-8895-ba8ce19ffe18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:18.454299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606699f-4f4d-44df-8895-ba8ce19ffe18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:18.455768 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606699f-4f4d-44df-8895-ba8ce19ffe18 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:18.467772 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1226/2431] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:19.148813 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49e1f0-192d-4965-b4a6-eabc3e43f09e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:19.150898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fc49e1f0-192d-4965-b4a6-eabc3e43f09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:19.247751 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49e1f0-192d-4965-b4a6-eabc3e43f09e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:19.260345 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57054c-445f-4ce1-a4fd-7e9f90169e3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd07165f554a445169b2c3f0d173c0bb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:19.261208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57054c-445f-4ce1-a4fd-7e9f90169e3a service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:19.267408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57054c-445f-4ce1-a4fd-7e9f90169e3a service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:19.268453 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57054c-445f-4ce1-a4fd-7e9f90169e3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:19.271316 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-be57054c-445f-4ce1-a4fd-7e9f90169e3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:19.291072 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57054c-445f-4ce1-a4fd-7e9f90169e3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:19.293083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57054c-445f-4ce1-a4fd-7e9f90169e3a service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:19.306682 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1206/2432] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:19.611054 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606699f-4f4d-44df-8895-ba8ce19ffe18 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:19.622514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606699f-4f4d-44df-8895-ba8ce19ffe18 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:19.632341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a8b37-c30a-4836-8f64-7c5fef057754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:19.633098 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a8b37-c30a-4836-8f64-7c5fef057754 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:19.633386 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a8b37-c30a-4836-8f64-7c5fef057754 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:19.633754 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a8b37-c30a-4836-8f64-7c5fef057754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:19.635758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-348a8b37-c30a-4836-8f64-7c5fef057754 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:19.648854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a8b37-c30a-4836-8f64-7c5fef057754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:19.650875 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a8b37-c30a-4836-8f64-7c5fef057754 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:19.683500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1227/2433] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:21.617745 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab3f3c-8e36-494e-a155-983a983afde7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:21.618313 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab3f3c-8e36-494e-a155-983a983afde7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:21.618313 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab3f3c-8e36-494e-a155-983a983afde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:21.642679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-92ab3f3c-8e36-494e-a155-983a983afde7 None None] MFA Rules not processed for user `f25c72896a2d4d619e5ad4b5a0a79aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:21.644434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-92ab3f3c-8e36-494e-a155-983a983afde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:21.670263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:21.750418 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1207/2434] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:34:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:26.863092 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d906a-4401-411b-859d-1b18ad6f667b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:26.863586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d906a-4401-411b-859d-1b18ad6f667b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:26.863720 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d906a-4401-411b-859d-1b18ad6f667b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:26.881006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b85d906a-4401-411b-859d-1b18ad6f667b None None] MFA Rules not processed for user `a09029144c454b099bea86aaa5229ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:26.882336 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b85d906a-4401-411b-859d-1b18ad6f667b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:26.942265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1228/2435] 10.222.0.33 () {58 vars in 967 bytes} [Mon Jul 18 10:34:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:26.990503 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab3f3c-8e36-494e-a155-983a983afde7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:26.991921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-92ab3f3c-8e36-494e-a155-983a983afde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:27.061730 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab3f3c-8e36-494e-a155-983a983afde7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:27.066776 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165de21-77f6-457f-9858-8adf965abcf3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6168873f2846c8af73b903bba17eb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:27.067425 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165de21-77f6-457f-9858-8adf965abcf3 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:27.067713 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165de21-77f6-457f-9858-8adf965abcf3 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:27.067992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165de21-77f6-457f-9858-8adf965abcf3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:27.076188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165de21-77f6-457f-9858-8adf965abcf3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:27.078318 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165de21-77f6-457f-9858-8adf965abcf3 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:27.086727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1208/2436] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 18 10:34:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:34:27.433554 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d906a-4401-411b-859d-1b18ad6f667b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:27.434993 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b85d906a-4401-411b-859d-1b18ad6f667b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:27.461786 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d906a-4401-411b-859d-1b18ad6f667b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:27.467908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1f0ab-add4-4c44-a6e0-f93b181d0029 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:27.468716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1f0ab-add4-4c44-a6e0-f93b181d0029 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:27.469072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1f0ab-add4-4c44-a6e0-f93b181d0029 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:27.469415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1f0ab-add4-4c44-a6e0-f93b181d0029 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:27.472027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2ca1f0ab-add4-4c44-a6e0-f93b181d0029 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:27.486740 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1f0ab-add4-4c44-a6e0-f93b181d0029 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:27.487979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1f0ab-add4-4c44-a6e0-f93b181d0029 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:27.499425 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1229/2437] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:34.407228 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165de21-77f6-457f-9858-8adf965abcf3 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:34.408744 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b165de21-77f6-457f-9858-8adf965abcf3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:34.423059 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165de21-77f6-457f-9858-8adf965abcf3 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:34.427595 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845926e-36c9-4d9c-89f5-dc51dd964d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:34.428464 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845926e-36c9-4d9c-89f5-dc51dd964d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:34.428797 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845926e-36c9-4d9c-89f5-dc51dd964d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:34.429078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845926e-36c9-4d9c-89f5-dc51dd964d85 admin admin] PATH_INFO: `/v3/users/1af30265de71409d876112350949a2a4` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:34.430301 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c845926e-36c9-4d9c-89f5-dc51dd964d85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1af30265de71409d876112350949a2a4)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:34.432224 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c845926e-36c9-4d9c-89f5-dc51dd964d85 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:34.477571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c845926e-36c9-4d9c-89f5-dc51dd964d85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1af30265de71409d876112350949a2a4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:34.478253 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c845926e-36c9-4d9c-89f5-dc51dd964d85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1af30265de71409d876112350949a2a4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:34.478865 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c845926e-36c9-4d9c-89f5-dc51dd964d85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1af30265de71409d876112350949a2a4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:34.486226 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c845926e-36c9-4d9c-89f5-dc51dd964d85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1af30265de71409d876112350949a2a4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:34.497768 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c845926e-36c9-4d9c-89f5-dc51dd964d85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1af30265de71409d876112350949a2a4'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:34.515422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1209/2438] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:34:34 2022] DELETE /identity/v3/users/1af30265de71409d876112350949a2a4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:34:34.697151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1f0ab-add4-4c44-a6e0-f93b181d0029 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:34.713314 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1f0ab-add4-4c44-a6e0-f93b181d0029 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:34.757352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:34.774703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be62d4e-9031-4629-92c3-baf7ee834f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:34.775573 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be62d4e-9031-4629-92c3-baf7ee834f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:34.775861 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be62d4e-9031-4629-92c3-baf7ee834f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:34.776145 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be62d4e-9031-4629-92c3-baf7ee834f92 admin admin] PATH_INFO: `/v3/projects/f62ef510c8484995a2f31f29104f399f` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:34.777363 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be62d4e-9031-4629-92c3-baf7ee834f92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f62ef510c8484995a2f31f29104f399f)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:34.779286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be62d4e-9031-4629-92c3-baf7ee834f92 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:34.828457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8be62d4e-9031-4629-92c3-baf7ee834f92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f62ef510c8484995a2f31f29104f399f', 'request_id': 'req-8be62d4e-9031-4629-92c3-baf7ee834f92'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:34.833967 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8be62d4e-9031-4629-92c3-baf7ee834f92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f62ef510c8484995a2f31f29104f399f', 'request_id': 'req-8be62d4e-9031-4629-92c3-baf7ee834f92'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:34.840038 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8be62d4e-9031-4629-92c3-baf7ee834f92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f62ef510c8484995a2f31f29104f399f', 'request_id': 'req-8be62d4e-9031-4629-92c3-baf7ee834f92'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:34.853404 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8be62d4e-9031-4629-92c3-baf7ee834f92 admin admin] The token cache is being invalidate because project f62ef510c8484995a2f31f29104f399f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:34:34.853695 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8be62d4e-9031-4629-92c3-baf7ee834f92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:34.854368 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8be62d4e-9031-4629-92c3-baf7ee834f92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:34.856206 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1230/2439] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:34:34 2022] DELETE /identity/v3/projects/f62ef510c8484995a2f31f29104f399f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:34:34.860138 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845926e-36c9-4d9c-89f5-dc51dd964d85 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:34.861873 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c845926e-36c9-4d9c-89f5-dc51dd964d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:34.875564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845926e-36c9-4d9c-89f5-dc51dd964d85 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:34.897293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:34.941390 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192134e6-9f88-4895-86a2-55aa3a2e11cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:34.942184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192134e6-9f88-4895-86a2-55aa3a2e11cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:34.942463 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192134e6-9f88-4895-86a2-55aa3a2e11cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:34.942735 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192134e6-9f88-4895-86a2-55aa3a2e11cb admin admin] PATH_INFO: `/v3/users/a2fc697f771d43b6a0f29f5adf8cb4d9` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:34.943932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192134e6-9f88-4895-86a2-55aa3a2e11cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2fc697f771d43b6a0f29f5adf8cb4d9)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:34.945919 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192134e6-9f88-4895-86a2-55aa3a2e11cb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:35.001488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-192134e6-9f88-4895-86a2-55aa3a2e11cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2fc697f771d43b6a0f29f5adf8cb4d9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:35.002173 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-192134e6-9f88-4895-86a2-55aa3a2e11cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2fc697f771d43b6a0f29f5adf8cb4d9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:35.002772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-192134e6-9f88-4895-86a2-55aa3a2e11cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2fc697f771d43b6a0f29f5adf8cb4d9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:35.010652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-192134e6-9f88-4895-86a2-55aa3a2e11cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2fc697f771d43b6a0f29f5adf8cb4d9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:35.023836 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-192134e6-9f88-4895-86a2-55aa3a2e11cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2fc697f771d43b6a0f29f5adf8cb4d9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:35.039248 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1210/2440] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:34:34 2022] DELETE /identity/v3/users/a2fc697f771d43b6a0f29f5adf8cb4d9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:34:35.231034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be62d4e-9031-4629-92c3-baf7ee834f92 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:35.233270 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8be62d4e-9031-4629-92c3-baf7ee834f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:35.255810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be62d4e-9031-4629-92c3-baf7ee834f92 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:35.360844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798367fc-510d-415d-b081-15a08563d6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:35.360844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798367fc-510d-415d-b081-15a08563d6a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:35.360844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798367fc-510d-415d-b081-15a08563d6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:35.360844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798367fc-510d-415d-b081-15a08563d6a4 admin admin] PATH_INFO: `/v3/projects/ddd22b027e204f30848e177ea4b8dfe6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:35.361535 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798367fc-510d-415d-b081-15a08563d6a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddd22b027e204f30848e177ea4b8dfe6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:35.361535 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798367fc-510d-415d-b081-15a08563d6a4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:35.411660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-798367fc-510d-415d-b081-15a08563d6a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddd22b027e204f30848e177ea4b8dfe6', 'request_id': 'req-798367fc-510d-415d-b081-15a08563d6a4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:35.417640 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-798367fc-510d-415d-b081-15a08563d6a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddd22b027e204f30848e177ea4b8dfe6', 'request_id': 'req-798367fc-510d-415d-b081-15a08563d6a4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:35.424491 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-798367fc-510d-415d-b081-15a08563d6a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddd22b027e204f30848e177ea4b8dfe6', 'request_id': 'req-798367fc-510d-415d-b081-15a08563d6a4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:35.437473 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-798367fc-510d-415d-b081-15a08563d6a4 admin admin] The token cache is being invalidate because project ddd22b027e204f30848e177ea4b8dfe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:34:35.437765 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-798367fc-510d-415d-b081-15a08563d6a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:35.438393 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-798367fc-510d-415d-b081-15a08563d6a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:34:35.441120 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1231/2441] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:34:35 2022] DELETE /identity/v3/projects/ddd22b027e204f30848e177ea4b8dfe6 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:34:35.445834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fc455-fef8-41a9-b787-82ec6d827553 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:35.446382 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fc455-fef8-41a9-b787-82ec6d827553 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:35.446493 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fc455-fef8-41a9-b787-82ec6d827553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:35.465412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-211fc455-fef8-41a9-b787-82ec6d827553 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:35.466613 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-211fc455-fef8-41a9-b787-82ec6d827553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:35.533624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1211/2442] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:34:35.541107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cde7e-80d3-47e3-85af-8dd49dd43dd8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:35.541508 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cde7e-80d3-47e3-85af-8dd49dd43dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:35.541886 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cde7e-80d3-47e3-85af-8dd49dd43dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:35.550941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:35.563986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d42cde7e-80d3-47e3-85af-8dd49dd43dd8 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:35.565518 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d42cde7e-80d3-47e3-85af-8dd49dd43dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:35.575046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1232/2443] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:34:35.579385 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fc455-fef8-41a9-b787-82ec6d827553 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:35.590512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fc455-fef8-41a9-b787-82ec6d827553 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:35.595836 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44cc9d-bd56-4260-a5c5-04642effa128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:35.596546 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44cc9d-bd56-4260-a5c5-04642effa128 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:35.596807 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44cc9d-bd56-4260-a5c5-04642effa128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:35.597193 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44cc9d-bd56-4260-a5c5-04642effa128 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:35.597953 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44cc9d-bd56-4260-a5c5-04642effa128 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:34:35.598259 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44cc9d-bd56-4260-a5c5-04642effa128 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:35.599855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44cc9d-bd56-4260-a5c5-04642effa128 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:35.612855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1212/2444] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:34:35.617352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cde7e-80d3-47e3-85af-8dd49dd43dd8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:35.626824 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cde7e-80d3-47e3-85af-8dd49dd43dd8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:35.631569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521dff88-ff1d-4943-98a6-dadd00bee84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:35.632257 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dff88-ff1d-4943-98a6-dadd00bee84b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:35.632572 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dff88-ff1d-4943-98a6-dadd00bee84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:35.632857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dff88-ff1d-4943-98a6-dadd00bee84b admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:35.633563 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521dff88-ff1d-4943-98a6-dadd00bee84b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:35.635522 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521dff88-ff1d-4943-98a6-dadd00bee84b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:35.664703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1233/2445] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:34:35 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:34:35.665882 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44cc9d-bd56-4260-a5c5-04642effa128 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:35.677408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44cc9d-bd56-4260-a5c5-04642effa128 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:35.712182 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:35.747018 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f20f4-4b0d-458d-89ab-514c85c86356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:35.747605 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f20f4-4b0d-458d-89ab-514c85c86356 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:35.747896 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f20f4-4b0d-458d-89ab-514c85c86356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:35.748179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f20f4-4b0d-458d-89ab-514c85c86356 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:35.748907 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f20f4-4b0d-458d-89ab-514c85c86356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:35.750861 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f20f4-4b0d-458d-89ab-514c85c86356 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:35.771979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1213/2446] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:34:35 2022] POST /identity/v3/users => generated 286 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:34:35.776310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521dff88-ff1d-4943-98a6-dadd00bee84b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:35.787518 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521dff88-ff1d-4943-98a6-dadd00bee84b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:35.792250 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3edbbc-9379-4068-bf73-c236ef080d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:35.792964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3edbbc-9379-4068-bf73-c236ef080d0c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:35.793253 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3edbbc-9379-4068-bf73-c236ef080d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:35.793860 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3edbbc-9379-4068-bf73-c236ef080d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:35.794256 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3edbbc-9379-4068-bf73-c236ef080d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:35.795866 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3edbbc-9379-4068-bf73-c236ef080d0c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:35.807211 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1234/2447] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:34:35.810939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f20f4-4b0d-458d-89ab-514c85c86356 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:35.823282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f20f4-4b0d-458d-89ab-514c85c86356 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:35.828632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb13106-3243-4046-95b6-7d2649b404c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:35.829294 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb13106-3243-4046-95b6-7d2649b404c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:35.829579 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb13106-3243-4046-95b6-7d2649b404c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:35.829857 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb13106-3243-4046-95b6-7d2649b404c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:35.830587 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb13106-3243-4046-95b6-7d2649b404c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:35.832184 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb13106-3243-4046-95b6-7d2649b404c4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:35.842261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1214/2448] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:34:35.847020 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3edbbc-9379-4068-bf73-c236ef080d0c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:35.857843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3edbbc-9379-4068-bf73-c236ef080d0c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:35.862563 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9385e-e8db-4b29-9b34-ec33db95fb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:35.863462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9385e-e8db-4b29-9b34-ec33db95fb97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:35.863745 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9385e-e8db-4b29-9b34-ec33db95fb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:35.864027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9385e-e8db-4b29-9b34-ec33db95fb97 admin admin] PATH_INFO: `/v3/projects/5628452a1ceb4af293ea96c187983fa6/users/de7190c1ae524da280700ef2548a4a17/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:35.878528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9385e-e8db-4b29-9b34-ec33db95fb97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5628452a1ceb4af293ea96c187983fa6, user_id=de7190c1ae524da280700ef2548a4a17, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:35.881247 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9385e-e8db-4b29-9b34-ec33db95fb97 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:35.895497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1235/2449] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:34:35 2022] PUT /identity/v3/projects/5628452a1ceb4af293ea96c187983fa6/users/de7190c1ae524da280700ef2548a4a17/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:34:35.901194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8a856-15a6-4079-929e-2d74fa613d52 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:35.901523 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8a856-15a6-4079-929e-2d74fa613d52 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:35.901902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8a856-15a6-4079-929e-2d74fa613d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:35.920135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e2a8a856-15a6-4079-929e-2d74fa613d52 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:35.921422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e2a8a856-15a6-4079-929e-2d74fa613d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:35.979855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1215/2450] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:34:35.995139 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9385e-e8db-4b29-9b34-ec33db95fb97 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:35.997007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8bc9385e-e8db-4b29-9b34-ec33db95fb97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:36.058714 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9385e-e8db-4b29-9b34-ec33db95fb97 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:36.062818 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d375da4-3ae9-4f35-94c9-9a259e4094ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:36.063457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d375da4-3ae9-4f35-94c9-9a259e4094ba service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:36.063739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d375da4-3ae9-4f35-94c9-9a259e4094ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:36.064017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d375da4-3ae9-4f35-94c9-9a259e4094ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:36.073526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d375da4-3ae9-4f35-94c9-9a259e4094ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:36.074947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d375da4-3ae9-4f35-94c9-9a259e4094ba service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:36.084543 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1236/2451] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:36.287423 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8a856-15a6-4079-929e-2d74fa613d52 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:36.289070 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e2a8a856-15a6-4079-929e-2d74fa613d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:36.398812 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8a856-15a6-4079-929e-2d74fa613d52 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:36.405810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffa82c-cf29-469e-afa8-fb87994ecb57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:36.406497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffa82c-cf29-469e-afa8-fb87994ecb57 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:36.406784 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffa82c-cf29-469e-afa8-fb87994ecb57 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:36.407578 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffa82c-cf29-469e-afa8-fb87994ecb57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:36.409647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-beffa82c-cf29-469e-afa8-fb87994ecb57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:36.425007 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffa82c-cf29-469e-afa8-fb87994ecb57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:36.426772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffa82c-cf29-469e-afa8-fb87994ecb57 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:36.454274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1216/2452] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:34:40.813167 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b7fda-9989-488e-ab81-bba08440c7c7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:40.813682 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b7fda-9989-488e-ab81-bba08440c7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:40.813797 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b7fda-9989-488e-ab81-bba08440c7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:40.842806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-348b7fda-9989-488e-ab81-bba08440c7c7 None None] MFA Rules not processed for user `de7190c1ae524da280700ef2548a4a17`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:40.844492 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-348b7fda-9989-488e-ab81-bba08440c7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:40.920076 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1237/2453] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:34:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:40.925140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffa82c-cf29-469e-afa8-fb87994ecb57 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:40.935958 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffa82c-cf29-469e-afa8-fb87994ecb57 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:40.940774 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9243c7-fe6d-4d65-9fe3-3ba404e37572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:40.941440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9243c7-fe6d-4d65-9fe3-3ba404e37572 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:40.941721 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9243c7-fe6d-4d65-9fe3-3ba404e37572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:40.941995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9243c7-fe6d-4d65-9fe3-3ba404e37572 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:40.942704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9243c7-fe6d-4d65-9fe3-3ba404e37572 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:40.944660 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9243c7-fe6d-4d65-9fe3-3ba404e37572 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:40.972644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1217/2454] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:34:40 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:34:40.976731 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b7fda-9989-488e-ab81-bba08440c7c7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:40.988478 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b7fda-9989-488e-ab81-bba08440c7c7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:41.020835 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:41.050116 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c8db1-554a-4cc7-9bba-0f7c6f248b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:41.050812 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c8db1-554a-4cc7-9bba-0f7c6f248b90 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:41.051640 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c8db1-554a-4cc7-9bba-0f7c6f248b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:41.051640 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c8db1-554a-4cc7-9bba-0f7c6f248b90 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:41.052233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c8db1-554a-4cc7-9bba-0f7c6f248b90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:41.054259 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c8db1-554a-4cc7-9bba-0f7c6f248b90 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:41.079720 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1238/2455] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:34:40 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:34:41.081083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9243c7-fe6d-4d65-9fe3-3ba404e37572 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:41.092098 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9243c7-fe6d-4d65-9fe3-3ba404e37572 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:41.097404 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1033c283-491f-4b79-a831-e5dab4eef0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:41.098082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1033c283-491f-4b79-a831-e5dab4eef0c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:41.098371 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1033c283-491f-4b79-a831-e5dab4eef0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:41.098657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1033c283-491f-4b79-a831-e5dab4eef0c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:41.099402 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1033c283-491f-4b79-a831-e5dab4eef0c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:41.101054 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1033c283-491f-4b79-a831-e5dab4eef0c8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:41.112755 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1218/2456] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:34:41.116919 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c8db1-554a-4cc7-9bba-0f7c6f248b90 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:41.127853 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c8db1-554a-4cc7-9bba-0f7c6f248b90 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:41.133043 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d301f7-567d-4d9d-895d-c91eb1facb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:41.133935 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d301f7-567d-4d9d-895d-c91eb1facb45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:41.134224 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d301f7-567d-4d9d-895d-c91eb1facb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:41.134504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d301f7-567d-4d9d-895d-c91eb1facb45 admin admin] PATH_INFO: `/v3/projects/a288f00152ee4a52945eb7be54d28207/users/6677d68e1f734f1ca65e0e25f6df3c12/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:41.150698 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d301f7-567d-4d9d-895d-c91eb1facb45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a288f00152ee4a52945eb7be54d28207, user_id=6677d68e1f734f1ca65e0e25f6df3c12, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:41.153446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d301f7-567d-4d9d-895d-c91eb1facb45 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:41.168158 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1239/2457] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:34:41 2022] PUT /identity/v3/projects/a288f00152ee4a52945eb7be54d28207/users/6677d68e1f734f1ca65e0e25f6df3c12/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:34:41.172003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1033c283-491f-4b79-a831-e5dab4eef0c8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:41.177322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:41.182769 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1033c283-491f-4b79-a831-e5dab4eef0c8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:41.243906 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517aa9e-19a0-4112-8bf2-e5151e4e7439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:41.244613 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517aa9e-19a0-4112-8bf2-e5151e4e7439 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:41.244904 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517aa9e-19a0-4112-8bf2-e5151e4e7439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:41.245182 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517aa9e-19a0-4112-8bf2-e5151e4e7439 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:41.245937 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517aa9e-19a0-4112-8bf2-e5151e4e7439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:41.247600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517aa9e-19a0-4112-8bf2-e5151e4e7439 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:41.257813 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1219/2458] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:34:41.262015 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d301f7-567d-4d9d-895d-c91eb1facb45 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:41.273532 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d301f7-567d-4d9d-895d-c91eb1facb45 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:41.278328 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78894f8e-81ff-4f68-932f-c77c25048696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:41.279215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78894f8e-81ff-4f68-932f-c77c25048696 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:41.279498 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78894f8e-81ff-4f68-932f-c77c25048696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:41.279777 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78894f8e-81ff-4f68-932f-c77c25048696 admin admin] PATH_INFO: `/v3/projects/a288f00152ee4a52945eb7be54d28207/users/6677d68e1f734f1ca65e0e25f6df3c12/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:41.282606 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78894f8e-81ff-4f68-932f-c77c25048696 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a288f00152ee4a52945eb7be54d28207, user_id=6677d68e1f734f1ca65e0e25f6df3c12, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:41.285563 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78894f8e-81ff-4f68-932f-c77c25048696 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:41.296408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1240/2459] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:34:41 2022] PUT /identity/v3/projects/a288f00152ee4a52945eb7be54d28207/users/6677d68e1f734f1ca65e0e25f6df3c12/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:34:45.509164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70169eff-c982-4738-b27c-84abeb56b5ba None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:45.509690 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70169eff-c982-4738-b27c-84abeb56b5ba None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:45.509810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70169eff-c982-4738-b27c-84abeb56b5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:45.536309 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-70169eff-c982-4738-b27c-84abeb56b5ba None None] MFA Rules not processed for user `6677d68e1f734f1ca65e0e25f6df3c12`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:45.537680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-70169eff-c982-4738-b27c-84abeb56b5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:45.608075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1220/2460] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:34:45 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:45.621889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78894f8e-81ff-4f68-932f-c77c25048696 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:34:45.699954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78894f8e-81ff-4f68-932f-c77c25048696 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:34:45.706008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0352a-1480-40d6-91ef-a7dfe6c9bfa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:34:45.706655 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0352a-1480-40d6-91ef-a7dfe6c9bfa1 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:45.706938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0352a-1480-40d6-91ef-a7dfe6c9bfa1 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:45.707213 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0352a-1480-40d6-91ef-a7dfe6c9bfa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:45.773449 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab0352a-1480-40d6-91ef-a7dfe6c9bfa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:34:45.775097 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab0352a-1480-40d6-91ef-a7dfe6c9bfa1 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:34:45.786118 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1241/2461] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:34:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:34:46.800591 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdbce4-4f2d-4634-91cd-7c47f89a37c1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:34:46.801072 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdbce4-4f2d-4634-91cd-7c47f89a37c1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:34:46.801187 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdbce4-4f2d-4634-91cd-7c47f89a37c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:34:46.822401 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-02fdbce4-4f2d-4634-91cd-7c47f89a37c1 None None] MFA Rules not processed for user `de7190c1ae524da280700ef2548a4a17`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:34:46.823647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-02fdbce4-4f2d-4634-91cd-7c47f89a37c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:34:46.831526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1221/2462] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:34:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:34:55.781682 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=85,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:34:56.816289 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=65 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:06.172177 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0352a-1480-40d6-91ef-a7dfe6c9bfa1 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:06.173665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4ab0352a-1480-40d6-91ef-a7dfe6c9bfa1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:06.189756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0352a-1480-40d6-91ef-a7dfe6c9bfa1 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:06.195364 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737d6a8-837c-455d-bca9-2a9ca207f653 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:06.196022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737d6a8-837c-455d-bca9-2a9ca207f653 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:06.196312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737d6a8-837c-455d-bca9-2a9ca207f653 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:06.196621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737d6a8-837c-455d-bca9-2a9ca207f653 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:06.198395 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3737d6a8-837c-455d-bca9-2a9ca207f653 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:06.263127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737d6a8-837c-455d-bca9-2a9ca207f653 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:06.264937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737d6a8-837c-455d-bca9-2a9ca207f653 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:06.275082 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1242/2463] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:15.805048 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fdbce4-4f2d-4634-91cd-7c47f89a37c1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:15.823994 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fdbce4-4f2d-4634-91cd-7c47f89a37c1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:15.830279 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0eba70-0520-46c1-a452-bc03f5f68f5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:15.831660 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0eba70-0520-46c1-a452-bc03f5f68f5e service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:15.831962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0eba70-0520-46c1-a452-bc03f5f68f5e service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:15.832365 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0eba70-0520-46c1-a452-bc03f5f68f5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:15.834619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4c0eba70-0520-46c1-a452-bc03f5f68f5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:15.851427 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0eba70-0520-46c1-a452-bc03f5f68f5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:15.853446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0eba70-0520-46c1-a452-bc03f5f68f5e service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:15.872854 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1222/2464] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:16.270515 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=28 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:18.547473 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737d6a8-837c-455d-bca9-2a9ca207f653 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:18.602629 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737d6a8-837c-455d-bca9-2a9ca207f653 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:18.606233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bb6bb-8f8e-4650-9ca5-11b7f1be146a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:18.606870 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bb6bb-8f8e-4650-9ca5-11b7f1be146a service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:18.607155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bb6bb-8f8e-4650-9ca5-11b7f1be146a service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:18.607432 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bb6bb-8f8e-4650-9ca5-11b7f1be146a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:18.616821 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635bb6bb-8f8e-4650-9ca5-11b7f1be146a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:18.618245 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635bb6bb-8f8e-4650-9ca5-11b7f1be146a service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:18.627479 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1243/2465] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:18 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:25.861328 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:28.623363 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=26 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:28.653716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5944f-f519-4d77-a84c-a414f15a6590 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:28.654223 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5944f-f519-4d77-a84c-a414f15a6590 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:28.654332 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5944f-f519-4d77-a84c-a414f15a6590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:28.671479 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-32a5944f-f519-4d77-a84c-a414f15a6590 None None] MFA Rules not processed for user `4e9eb8d188e040bca894e038955f2402`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:28.672929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-32a5944f-f519-4d77-a84c-a414f15a6590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:28.728018 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1223/2466] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:35:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:29.044051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bb6bb-8f8e-4650-9ca5-11b7f1be146a service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:29.053286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bb6bb-8f8e-4650-9ca5-11b7f1be146a service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:29.056903 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91e713-605f-4075-87f6-0bb2f5641f1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:29.057538 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91e713-605f-4075-87f6-0bb2f5641f1f service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:29.057825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91e713-605f-4075-87f6-0bb2f5641f1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:29.058100 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91e713-605f-4075-87f6-0bb2f5641f1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:29.059856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ff91e713-605f-4075-87f6-0bb2f5641f1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:29.069205 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91e713-605f-4075-87f6-0bb2f5641f1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:29.070614 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91e713-605f-4075-87f6-0bb2f5641f1f service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:29.078467 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1244/2467] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:29.929998 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ee72c-c4ec-47b3-b98e-3d9aa92f446c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:29.930946 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ee72c-c4ec-47b3-b98e-3d9aa92f446c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:29.931404 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ee72c-c4ec-47b3-b98e-3d9aa92f446c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:29.950172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b05ee72c-c4ec-47b3-b98e-3d9aa92f446c None None] MFA Rules not processed for user `a09029144c454b099bea86aaa5229ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:29.951398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b05ee72c-c4ec-47b3-b98e-3d9aa92f446c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:30.020077 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1224/2468] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 18 10:35:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:30.076971 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91e713-605f-4075-87f6-0bb2f5641f1f service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:30.078563 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ff91e713-605f-4075-87f6-0bb2f5641f1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:30.155383 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91e713-605f-4075-87f6-0bb2f5641f1f service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:30.162864 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8528b-5467-4b96-9281-d3055a482084 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6168873f2846c8af73b903bba17eb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:30.163522 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8528b-5467-4b96-9281-d3055a482084 service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:30.163811 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8528b-5467-4b96-9281-d3055a482084 service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:30.164093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8528b-5467-4b96-9281-d3055a482084 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:30.172926 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8528b-5467-4b96-9281-d3055a482084 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:30.174769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8528b-5467-4b96-9281-d3055a482084 service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:30.184272 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1245/2469] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 18 10:35:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:35:40.012557 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=50 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:40.180862 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=38 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:46.367888 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba14aa-145b-461b-96e2-38cbd66ef7b8 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:46.368447 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba14aa-145b-461b-96e2-38cbd66ef7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:46.368447 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba14aa-145b-461b-96e2-38cbd66ef7b8 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:46.369946 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1225/2470] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 18 10:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:46.376276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999ccea-66f6-42f6-9ba9-d7ca03b9ebd7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:46.376818 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999ccea-66f6-42f6-9ba9-d7ca03b9ebd7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:46.376915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999ccea-66f6-42f6-9ba9-d7ca03b9ebd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:46.396592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c999ccea-66f6-42f6-9ba9-d7ca03b9ebd7 None None] MFA Rules not processed for user `29048c2021e340b081ae44a390583e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:46.398266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c999ccea-66f6-42f6-9ba9-d7ca03b9ebd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:46.410073 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1246/2471] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 18 10:35:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:46.423691 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba14aa-145b-461b-96e2-38cbd66ef7b8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:46.425656 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-60ba14aa-145b-461b-96e2-38cbd66ef7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:46.513242 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba14aa-145b-461b-96e2-38cbd66ef7b8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:46.517248 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953792f-aa2c-4a03-a428-787cbe6a6871 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cee4d23797e430e9eeddc394ed7f50e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:46.518149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953792f-aa2c-4a03-a428-787cbe6a6871 service glance] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:46.518593 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953792f-aa2c-4a03-a428-787cbe6a6871 service glance] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:46.519010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953792f-aa2c-4a03-a428-787cbe6a6871 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:46.530951 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953792f-aa2c-4a03-a428-787cbe6a6871 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:46.532416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953792f-aa2c-4a03-a428-787cbe6a6871 service glance] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:46.545927 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1226/2472] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:46.633377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c896e-6fe6-4b57-b54e-27ea0ead6633 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:46.633606 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c896e-6fe6-4b57-b54e-27ea0ead6633 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:46.633898 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c896e-6fe6-4b57-b54e-27ea0ead6633 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:46.635279 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1247/2473] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 18 10:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 10:35:46.640628 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed1b54-c1df-4549-b77d-c3d4891defb6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:46.641009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed1b54-c1df-4549-b77d-c3d4891defb6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:46.641194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed1b54-c1df-4549-b77d-c3d4891defb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:46.661408 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-61ed1b54-c1df-4549-b77d-c3d4891defb6 None None] MFA Rules not processed for user `10d29b7934374f34923587eda9be0ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:46.662689 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-61ed1b54-c1df-4549-b77d-c3d4891defb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:46.670069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1227/2474] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 18 10:35:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:46.683542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c896e-6fe6-4b57-b54e-27ea0ead6633 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:46.696955 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c896e-6fe6-4b57-b54e-27ea0ead6633 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:46.703904 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9e8d6-8728-4e7f-bc13-0af75721b591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:46.704640 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9e8d6-8728-4e7f-bc13-0af75721b591 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:46.704926 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9e8d6-8728-4e7f-bc13-0af75721b591 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:46.705202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9e8d6-8728-4e7f-bc13-0af75721b591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:46.716046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9e8d6-8728-4e7f-bc13-0af75721b591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:46.717715 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9e8d6-8728-4e7f-bc13-0af75721b591 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:46.737801 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1248/2475] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:46.900050 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdbfb2-c47c-4e5f-8441-3ab32e38b388 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:46.900544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdbfb2-c47c-4e5f-8441-3ab32e38b388 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:46.900874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdbfb2-c47c-4e5f-8441-3ab32e38b388 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:46.902513 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1228/2476] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jul 18 10:35:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 10:35:46.915437 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5924a-05c0-4133-b812-f53003f5ecc5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:46.916173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5924a-05c0-4133-b812-f53003f5ecc5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:46.916457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5924a-05c0-4133-b812-f53003f5ecc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:46.939886 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b6f5924a-05c0-4133-b812-f53003f5ecc5 None None] MFA Rules not processed for user `29048c2021e340b081ae44a390583e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:46.941124 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b6f5924a-05c0-4133-b812-f53003f5ecc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:46.955948 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1249/2477] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jul 18 10:35:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:46.973691 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdbfb2-c47c-4e5f-8441-3ab32e38b388 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:46.987336 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdbfb2-c47c-4e5f-8441-3ab32e38b388 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:46.991205 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae51ea2-1e29-49de-92f4-76a2a7ce45ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:46.992033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae51ea2-1e29-49de-92f4-76a2a7ce45ec service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:46.993192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae51ea2-1e29-49de-92f4-76a2a7ce45ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:46.994388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae51ea2-1e29-49de-92f4-76a2a7ce45ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:47.008967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae51ea2-1e29-49de-92f4-76a2a7ce45ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:47.010710 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae51ea2-1e29-49de-92f4-76a2a7ce45ec service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:47.022574 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1229/2478] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:49.821094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae96947-84f5-4dae-a0c2-70e008f0e154 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:49.821564 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae96947-84f5-4dae-a0c2-70e008f0e154 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:49.821845 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae96947-84f5-4dae-a0c2-70e008f0e154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:49.842246 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4ae96947-84f5-4dae-a0c2-70e008f0e154 None None] MFA Rules not processed for user `a09029144c454b099bea86aaa5229ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:49.843740 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4ae96947-84f5-4dae-a0c2-70e008f0e154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:49.853611 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1250/2479] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 18 10:35:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:49.867379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae51ea2-1e29-49de-92f4-76a2a7ce45ec service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:49.880970 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae51ea2-1e29-49de-92f4-76a2a7ce45ec service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:49.886282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc48e49-0629-431e-a9fe-da799f0dcc97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6168873f2846c8af73b903bba17eb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:49.886954 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc48e49-0629-431e-a9fe-da799f0dcc97 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:49.887239 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc48e49-0629-431e-a9fe-da799f0dcc97 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:49.887509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc48e49-0629-431e-a9fe-da799f0dcc97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:49.897052 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc48e49-0629-431e-a9fe-da799f0dcc97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:49.898641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc48e49-0629-431e-a9fe-da799f0dcc97 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:49.909746 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1230/2480] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 18 10:35:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 10:35:50.828987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae96947-84f5-4dae-a0c2-70e008f0e154 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:50.850652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae96947-84f5-4dae-a0c2-70e008f0e154 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:50.862321 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98684c-2e36-4c74-80d4-8024e6b580f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:50.863134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98684c-2e36-4c74-80d4-8024e6b580f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:50.863515 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98684c-2e36-4c74-80d4-8024e6b580f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:50.864035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98684c-2e36-4c74-80d4-8024e6b580f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:50.866567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8e98684c-2e36-4c74-80d4-8024e6b580f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:51.007664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98684c-2e36-4c74-80d4-8024e6b580f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:51.011164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98684c-2e36-4c74-80d4-8024e6b580f3 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:51.028109 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1251/2481] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:52.074076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc48e49-0629-431e-a9fe-da799f0dcc97 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:52.075676 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dcc48e49-0629-431e-a9fe-da799f0dcc97 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:52.093581 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc48e49-0629-431e-a9fe-da799f0dcc97 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:52.118203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bd1b8-3312-4491-a3d9-b80b0c565f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:52.121065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bd1b8-3312-4491-a3d9-b80b0c565f3b service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:52.121467 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bd1b8-3312-4491-a3d9-b80b0c565f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:52.121850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bd1b8-3312-4491-a3d9-b80b0c565f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:52.129105 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-058bd1b8-3312-4491-a3d9-b80b0c565f3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:52.163985 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bd1b8-3312-4491-a3d9-b80b0c565f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:52.165823 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bd1b8-3312-4491-a3d9-b80b0c565f3b service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:52.195815 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1231/2482] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:57.906250 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98684c-2e36-4c74-80d4-8024e6b580f3 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:57.908052 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8e98684c-2e36-4c74-80d4-8024e6b580f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:58.033269 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98684c-2e36-4c74-80d4-8024e6b580f3 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:58.038617 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9cbc7-caa2-48c7-a450-70086c007a4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd07165f554a445169b2c3f0d173c0bb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:58.039505 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9cbc7-caa2-48c7-a450-70086c007a4f service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:58.040101 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9cbc7-caa2-48c7-a450-70086c007a4f service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:58.040603 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9cbc7-caa2-48c7-a450-70086c007a4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:58.042790 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3dc9cbc7-caa2-48c7-a450-70086c007a4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:58.068897 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9cbc7-caa2-48c7-a450-70086c007a4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:58.071225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9cbc7-caa2-48c7-a450-70086c007a4f service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:58.080006 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bd1b8-3312-4491-a3d9-b80b0c565f3b service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:58.082705 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-058bd1b8-3312-4491-a3d9-b80b0c565f3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:58.116981 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bd1b8-3312-4491-a3d9-b80b0c565f3b service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:58.119215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1252/2483] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:58.131986 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652e0e7-2996-477b-b37c-f8fb4bc6f9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:58.133727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652e0e7-2996-477b-b37c-f8fb4bc6f9b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:58.134263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652e0e7-2996-477b-b37c-f8fb4bc6f9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:58.134878 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652e0e7-2996-477b-b37c-f8fb4bc6f9b1 admin admin] PATH_INFO: `/v3/users/986f90dfbad14324b3fffbd790d60a37` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:58.141314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652e0e7-2996-477b-b37c-f8fb4bc6f9b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=986f90dfbad14324b3fffbd790d60a37)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:58.143297 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652e0e7-2996-477b-b37c-f8fb4bc6f9b1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:58.291987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7652e0e7-2996-477b-b37c-f8fb4bc6f9b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '986f90dfbad14324b3fffbd790d60a37'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:58.298273 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7652e0e7-2996-477b-b37c-f8fb4bc6f9b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '986f90dfbad14324b3fffbd790d60a37'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:58.299394 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7652e0e7-2996-477b-b37c-f8fb4bc6f9b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '986f90dfbad14324b3fffbd790d60a37'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:58.313871 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7652e0e7-2996-477b-b37c-f8fb4bc6f9b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '986f90dfbad14324b3fffbd790d60a37'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:58.362046 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7652e0e7-2996-477b-b37c-f8fb4bc6f9b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '986f90dfbad14324b3fffbd790d60a37'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:58.395656 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1232/2484] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:35:58 2022] DELETE /identity/v3/users/986f90dfbad14324b3fffbd790d60a37 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:35:58.411432 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9cbc7-caa2-48c7-a450-70086c007a4f service placement] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:58.432644 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9cbc7-caa2-48c7-a450-70086c007a4f service placement] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:58.551900 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652e0e7-2996-477b-b37c-f8fb4bc6f9b1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:58.553360 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7652e0e7-2996-477b-b37c-f8fb4bc6f9b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:58.562655 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:58.603539 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68478a-868a-4673-9262-f374a6ddca9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd07165f554a445169b2c3f0d173c0bb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:58.605844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68478a-868a-4673-9262-f374a6ddca9b service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:58.606949 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68478a-868a-4673-9262-f374a6ddca9b service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:58.608281 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68478a-868a-4673-9262-f374a6ddca9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:58.614182 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0e68478a-868a-4673-9262-f374a6ddca9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:58.627693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:58.660983 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e68478a-868a-4673-9262-f374a6ddca9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:58.662609 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e68478a-868a-4673-9262-f374a6ddca9b service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:58.682131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652e0e7-2996-477b-b37c-f8fb4bc6f9b1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:58.687450 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00308783-3646-47f6-b334-d0fa20ed908c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:58.688344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00308783-3646-47f6-b334-d0fa20ed908c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:58.688767 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1253/2485] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:35:58.696861 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00308783-3646-47f6-b334-d0fa20ed908c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:58.697183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00308783-3646-47f6-b334-d0fa20ed908c admin admin] PATH_INFO: `/v3/users/f25c72896a2d4d619e5ad4b5a0a79aa6` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:58.698569 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00308783-3646-47f6-b334-d0fa20ed908c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f25c72896a2d4d619e5ad4b5a0a79aa6)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:58.700567 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00308783-3646-47f6-b334-d0fa20ed908c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:58.795179 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-00308783-3646-47f6-b334-d0fa20ed908c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f25c72896a2d4d619e5ad4b5a0a79aa6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:58.796706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-00308783-3646-47f6-b334-d0fa20ed908c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f25c72896a2d4d619e5ad4b5a0a79aa6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:58.797716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-00308783-3646-47f6-b334-d0fa20ed908c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f25c72896a2d4d619e5ad4b5a0a79aa6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:58.810533 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68478a-868a-4673-9262-f374a6ddca9b service placement] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:58.812264 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0e68478a-868a-4673-9262-f374a6ddca9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:58.812719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-00308783-3646-47f6-b334-d0fa20ed908c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f25c72896a2d4d619e5ad4b5a0a79aa6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:58.834964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68478a-868a-4673-9262-f374a6ddca9b service placement] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:58.837480 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-00308783-3646-47f6-b334-d0fa20ed908c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f25c72896a2d4d619e5ad4b5a0a79aa6'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:58.856587 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1233/2486] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:35:58 2022] DELETE /identity/v3/users/f25c72896a2d4d619e5ad4b5a0a79aa6 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:35:58.860071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b750ea-ccff-4a91-a903-39907a9ac034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:58.861008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b750ea-ccff-4a91-a903-39907a9ac034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:58.861325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b750ea-ccff-4a91-a903-39907a9ac034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:58.862685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b750ea-ccff-4a91-a903-39907a9ac034 admin admin] PATH_INFO: `/v3/projects/e6e487deb7ff45bdbf2cc002d42d12a5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:58.864032 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b750ea-ccff-4a91-a903-39907a9ac034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6e487deb7ff45bdbf2cc002d42d12a5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:58.865999 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b750ea-ccff-4a91-a903-39907a9ac034 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:58.927064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-03b750ea-ccff-4a91-a903-39907a9ac034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6e487deb7ff45bdbf2cc002d42d12a5', 'request_id': 'req-03b750ea-ccff-4a91-a903-39907a9ac034'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:58.937749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-03b750ea-ccff-4a91-a903-39907a9ac034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6e487deb7ff45bdbf2cc002d42d12a5', 'request_id': 'req-03b750ea-ccff-4a91-a903-39907a9ac034'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:58.948416 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-03b750ea-ccff-4a91-a903-39907a9ac034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6e487deb7ff45bdbf2cc002d42d12a5', 'request_id': 'req-03b750ea-ccff-4a91-a903-39907a9ac034'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:58.964944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-03b750ea-ccff-4a91-a903-39907a9ac034 admin admin] The token cache is being invalidate because project e6e487deb7ff45bdbf2cc002d42d12a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:35:58.965246 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-03b750ea-ccff-4a91-a903-39907a9ac034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:58.965897 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-03b750ea-ccff-4a91-a903-39907a9ac034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:58.967618 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1254/2487] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:35:58 2022] DELETE /identity/v3/projects/e6e487deb7ff45bdbf2cc002d42d12a5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:35:58.972547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635167dc-76ae-4f94-b17e-8e9881feb099 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:58.972846 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635167dc-76ae-4f94-b17e-8e9881feb099 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:58.973132 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635167dc-76ae-4f94-b17e-8e9881feb099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:58.992182 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-635167dc-76ae-4f94-b17e-8e9881feb099 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:58.993444 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-635167dc-76ae-4f94-b17e-8e9881feb099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:59.055447 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1234/2488] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:59.063493 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c95ad1-967c-496d-8387-b33af7a4ccd0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.063843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c95ad1-967c-496d-8387-b33af7a4ccd0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.064127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c95ad1-967c-496d-8387-b33af7a4ccd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.080991 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635167dc-76ae-4f94-b17e-8e9881feb099 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.082474 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-635167dc-76ae-4f94-b17e-8e9881feb099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:59.086861 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-70c95ad1-967c-496d-8387-b33af7a4ccd0 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:59.088764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-70c95ad1-967c-496d-8387-b33af7a4ccd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:59.095877 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635167dc-76ae-4f94-b17e-8e9881feb099 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:59.098659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1255/2489] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:59.103513 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c95ad1-967c-496d-8387-b33af7a4ccd0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.104533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10922fdd-d2a5-4068-8eb4-42e3ff3356ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:59.105404 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10922fdd-d2a5-4068-8eb4-42e3ff3356ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.110461 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10922fdd-d2a5-4068-8eb4-42e3ff3356ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.110843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10922fdd-d2a5-4068-8eb4-42e3ff3356ea admin admin] PATH_INFO: `/v3/projects/1cf3d5bfe9a745f38372c42c9cfa1f63` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.114774 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10922fdd-d2a5-4068-8eb4-42e3ff3356ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cf3d5bfe9a745f38372c42c9cfa1f63)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:59.116677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c95ad1-967c-496d-8387-b33af7a4ccd0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:59.117165 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10922fdd-d2a5-4068-8eb4-42e3ff3356ea admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:59.121720 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e1d5d-79db-4d13-9422-cb60b6c88461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:59.122473 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e1d5d-79db-4d13-9422-cb60b6c88461 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.122827 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e1d5d-79db-4d13-9422-cb60b6c88461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.123174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e1d5d-79db-4d13-9422-cb60b6c88461 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.123981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e1d5d-79db-4d13-9422-cb60b6c88461 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:35:59.124418 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e1d5d-79db-4d13-9422-cb60b6c88461 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:59.126117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e1d5d-79db-4d13-9422-cb60b6c88461 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:59.138519 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1256/2490] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:35:59.144033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e1d5d-79db-4d13-9422-cb60b6c88461 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.157168 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e1d5d-79db-4d13-9422-cb60b6c88461 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:59.162352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97955af5-4216-4602-995a-0a3e7474d5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:59.163016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97955af5-4216-4602-995a-0a3e7474d5ac admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.163305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97955af5-4216-4602-995a-0a3e7474d5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.163588 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97955af5-4216-4602-995a-0a3e7474d5ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.164298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97955af5-4216-4602-995a-0a3e7474d5ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:59.166278 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97955af5-4216-4602-995a-0a3e7474d5ac admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:59.169249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:59.173477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-10922fdd-d2a5-4068-8eb4-42e3ff3356ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cf3d5bfe9a745f38372c42c9cfa1f63', 'request_id': 'req-10922fdd-d2a5-4068-8eb4-42e3ff3356ea'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:59.179728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-10922fdd-d2a5-4068-8eb4-42e3ff3356ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cf3d5bfe9a745f38372c42c9cfa1f63', 'request_id': 'req-10922fdd-d2a5-4068-8eb4-42e3ff3356ea'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:59.186534 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-10922fdd-d2a5-4068-8eb4-42e3ff3356ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cf3d5bfe9a745f38372c42c9cfa1f63', 'request_id': 'req-10922fdd-d2a5-4068-8eb4-42e3ff3356ea'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:59.196527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1257/2491] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:35:59 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:35:59.200622 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97955af5-4216-4602-995a-0a3e7474d5ac admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.202828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-10922fdd-d2a5-4068-8eb4-42e3ff3356ea admin admin] The token cache is being invalidate because project 1cf3d5bfe9a745f38372c42c9cfa1f63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:35:59.203126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-10922fdd-d2a5-4068-8eb4-42e3ff3356ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:59.203767 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-10922fdd-d2a5-4068-8eb4-42e3ff3356ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:35:59.205571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1235/2492] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:35:59 2022] DELETE /identity/v3/projects/1cf3d5bfe9a745f38372c42c9cfa1f63 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:35:59.210855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3c2c9-9652-4747-b526-2c34934bbe29 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.211151 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3c2c9-9652-4747-b526-2c34934bbe29 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.211496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3c2c9-9652-4747-b526-2c34934bbe29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.239362 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a2c3c2c9-9652-4747-b526-2c34934bbe29 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:59.240648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a2c3c2c9-9652-4747-b526-2c34934bbe29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:59.271019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97955af5-4216-4602-995a-0a3e7474d5ac admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:59.280383 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0ce7d-5381-4e48-8f54-4f204284d072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:59.281608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0ce7d-5381-4e48-8f54-4f204284d072 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.281900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0ce7d-5381-4e48-8f54-4f204284d072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.282186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0ce7d-5381-4e48-8f54-4f204284d072 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.283097 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0ce7d-5381-4e48-8f54-4f204284d072 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:59.284929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0ce7d-5381-4e48-8f54-4f204284d072 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:59.315288 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1258/2493] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:35:59 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:35:59.317640 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1236/2494] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:35:59.325303 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0ce7d-5381-4e48-8f54-4f204284d072 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.325687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4912cdd-7a32-4ae2-8ffd-92121c2a2076 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.326057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4912cdd-7a32-4ae2-8ffd-92121c2a2076 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.326425 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4912cdd-7a32-4ae2-8ffd-92121c2a2076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.327343 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-10f0ce7d-5381-4e48-8f54-4f204284d072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:59.340360 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0ce7d-5381-4e48-8f54-4f204284d072 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:59.345511 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d4912cdd-7a32-4ae2-8ffd-92121c2a2076 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:59.346756 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d4912cdd-7a32-4ae2-8ffd-92121c2a2076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:59.349583 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e409797-b508-43ae-ac25-6be01d5d29a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:59.350329 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e409797-b508-43ae-ac25-6be01d5d29a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.350614 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e409797-b508-43ae-ac25-6be01d5d29a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.350891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e409797-b508-43ae-ac25-6be01d5d29a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.352016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e409797-b508-43ae-ac25-6be01d5d29a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:59.353744 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e409797-b508-43ae-ac25-6be01d5d29a3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:59.356005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1237/2495] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:35:59.359633 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:59.359999 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4912cdd-7a32-4ae2-8ffd-92121c2a2076 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.367250 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1259/2496] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:35:59.371958 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4912cdd-7a32-4ae2-8ffd-92121c2a2076 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:59.377845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2843e-ccb2-447a-8a34-5a9e62777c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:59.379183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2843e-ccb2-447a-8a34-5a9e62777c63 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.379333 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e409797-b508-43ae-ac25-6be01d5d29a3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.379491 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2843e-ccb2-447a-8a34-5a9e62777c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.379982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2843e-ccb2-447a-8a34-5a9e62777c63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.380753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2843e-ccb2-447a-8a34-5a9e62777c63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:35:59.381072 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2843e-ccb2-447a-8a34-5a9e62777c63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:59.382655 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2843e-ccb2-447a-8a34-5a9e62777c63 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:59.395831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1238/2497] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:35:59.396785 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e409797-b508-43ae-ac25-6be01d5d29a3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:59.402287 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1f5e6-4c22-4cf7-8ff8-1c795e8ed4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:59.403048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1f5e6-4c22-4cf7-8ff8-1c795e8ed4ca admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.403340 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1f5e6-4c22-4cf7-8ff8-1c795e8ed4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.403802 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1f5e6-4c22-4cf7-8ff8-1c795e8ed4ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.404583 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1f5e6-4c22-4cf7-8ff8-1c795e8ed4ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:59.405559 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2843e-ccb2-447a-8a34-5a9e62777c63 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.406209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1f5e6-4c22-4cf7-8ff8-1c795e8ed4ca admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:59.416458 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2843e-ccb2-447a-8a34-5a9e62777c63 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:59.418821 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1260/2498] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:35:59.422836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1f5e6-4c22-4cf7-8ff8-1c795e8ed4ca admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.424628 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd1526-0466-48e0-9618-4400d6252ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:59.425422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd1526-0466-48e0-9618-4400d6252ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.425718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd1526-0466-48e0-9618-4400d6252ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.426087 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd1526-0466-48e0-9618-4400d6252ee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.426796 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd1526-0466-48e0-9618-4400d6252ee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:59.428972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd1526-0466-48e0-9618-4400d6252ee7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:59.433642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1f5e6-4c22-4cf7-8ff8-1c795e8ed4ca admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:59.441103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c154766-4c91-4d3e-a0a6-e4f0eae2ce06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:59.442163 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c154766-4c91-4d3e-a0a6-e4f0eae2ce06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.442565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c154766-4c91-4d3e-a0a6-e4f0eae2ce06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.442930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c154766-4c91-4d3e-a0a6-e4f0eae2ce06 admin admin] PATH_INFO: `/v3/projects/06860d35e56541e89e34db060cae0347/users/6960f1d6ff0747d3addabd96dc6df651/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.459266 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1239/2499] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:35:59 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:35:59.461630 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c154766-4c91-4d3e-a0a6-e4f0eae2ce06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06860d35e56541e89e34db060cae0347, user_id=6960f1d6ff0747d3addabd96dc6df651, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:59.463006 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd1526-0466-48e0-9618-4400d6252ee7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.464346 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c154766-4c91-4d3e-a0a6-e4f0eae2ce06 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:59.478761 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1261/2500] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:35:59 2022] PUT /identity/v3/projects/06860d35e56541e89e34db060cae0347/users/6960f1d6ff0747d3addabd96dc6df651/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:35:59.489379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f3fc8-4235-47aa-9cf8-75716fec0520 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.489873 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f3fc8-4235-47aa-9cf8-75716fec0520 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.490100 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f3fc8-4235-47aa-9cf8-75716fec0520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.523542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6d1f3fc8-4235-47aa-9cf8-75716fec0520 None None] MFA Rules not processed for user `6960f1d6ff0747d3addabd96dc6df651`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:59.524817 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6d1f3fc8-4235-47aa-9cf8-75716fec0520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:59.539210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd1526-0466-48e0-9618-4400d6252ee7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:59.582859 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:59.599831 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1262/2501] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:35:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:35:59.604586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f3fc8-4235-47aa-9cf8-75716fec0520 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.609233 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df880b29-ba63-4cef-93fc-e3c3ee6b2338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:59.609905 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df880b29-ba63-4cef-93fc-e3c3ee6b2338 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.610186 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df880b29-ba63-4cef-93fc-e3c3ee6b2338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.610459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df880b29-ba63-4cef-93fc-e3c3ee6b2338 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.611147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df880b29-ba63-4cef-93fc-e3c3ee6b2338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:59.613151 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df880b29-ba63-4cef-93fc-e3c3ee6b2338 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:59.634060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1240/2502] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:35:59 2022] POST /identity/v3/users => generated 284 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:35:59.638939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df880b29-ba63-4cef-93fc-e3c3ee6b2338 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.649203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df880b29-ba63-4cef-93fc-e3c3ee6b2338 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:59.654440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c7aae-5ae1-433e-aa72-88d88c244380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:59.655093 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c7aae-5ae1-433e-aa72-88d88c244380 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.655378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c7aae-5ae1-433e-aa72-88d88c244380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.655654 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c7aae-5ae1-433e-aa72-88d88c244380 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.656376 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c7aae-5ae1-433e-aa72-88d88c244380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:59.658267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c7aae-5ae1-433e-aa72-88d88c244380 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:59.669498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1241/2503] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:35:59.670507 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f3fc8-4235-47aa-9cf8-75716fec0520 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:59.673287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c7aae-5ae1-433e-aa72-88d88c244380 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.676270 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38783da-1051-4eb6-ac3c-07ec431c6726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:59.677053 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38783da-1051-4eb6-ac3c-07ec431c6726 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.677409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38783da-1051-4eb6-ac3c-07ec431c6726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.677753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38783da-1051-4eb6-ac3c-07ec431c6726 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.678516 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38783da-1051-4eb6-ac3c-07ec431c6726 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:59.680496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38783da-1051-4eb6-ac3c-07ec431c6726 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:59.684225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c7aae-5ae1-433e-aa72-88d88c244380 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:59.689067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15eee16-1b79-4a4b-8d26-17ae83ad886f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:59.690288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15eee16-1b79-4a4b-8d26-17ae83ad886f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.690580 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15eee16-1b79-4a4b-8d26-17ae83ad886f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.690861 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15eee16-1b79-4a4b-8d26-17ae83ad886f admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.691590 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15eee16-1b79-4a4b-8d26-17ae83ad886f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:59.693274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15eee16-1b79-4a4b-8d26-17ae83ad886f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:59.706360 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1242/2504] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:35:59.710223 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15eee16-1b79-4a4b-8d26-17ae83ad886f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.714870 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1263/2505] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:35:59 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:35:59.716309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38783da-1051-4eb6-ac3c-07ec431c6726 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.720804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15eee16-1b79-4a4b-8d26-17ae83ad886f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:59.766392 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:35:59.788050 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0ae65-60ea-4e30-bc17-e8e611ecf534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:59.788988 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0ae65-60ea-4e30-bc17-e8e611ecf534 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.789268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0ae65-60ea-4e30-bc17-e8e611ecf534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.789549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0ae65-60ea-4e30-bc17-e8e611ecf534 admin admin] PATH_INFO: `/v3/projects/43169a2c7a3b4545b6770fa990652739/users/bb5e75c6aae84a8d857a4377c22a1ba1/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.791120 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38783da-1051-4eb6-ac3c-07ec431c6726 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:59.795918 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7d174-ff72-4605-8ba3-680706e2aba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:59.796614 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7d174-ff72-4605-8ba3-680706e2aba1 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.796902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7d174-ff72-4605-8ba3-680706e2aba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.797178 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7d174-ff72-4605-8ba3-680706e2aba1 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.797866 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe7d174-ff72-4605-8ba3-680706e2aba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:59.799885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe7d174-ff72-4605-8ba3-680706e2aba1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:59.805318 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0ae65-60ea-4e30-bc17-e8e611ecf534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43169a2c7a3b4545b6770fa990652739, user_id=bb5e75c6aae84a8d857a4377c22a1ba1, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:59.807983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0ae65-60ea-4e30-bc17-e8e611ecf534 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:59.825720 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1264/2506] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:35:59 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:35:59.825950 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1243/2507] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:35:59 2022] PUT /identity/v3/projects/43169a2c7a3b4545b6770fa990652739/users/bb5e75c6aae84a8d857a4377c22a1ba1/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:35:59.827145 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7d174-ff72-4605-8ba3-680706e2aba1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.834985 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32480c-a715-4dc4-b726-864403e1f9d6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.835205 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32480c-a715-4dc4-b726-864403e1f9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.835480 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32480c-a715-4dc4-b726-864403e1f9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.837052 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7d174-ff72-4605-8ba3-680706e2aba1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:59.862995 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9e32480c-a715-4dc4-b726-864403e1f9d6 None None] MFA Rules not processed for user `bb5e75c6aae84a8d857a4377c22a1ba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:35:59.864220 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9e32480c-a715-4dc4-b726-864403e1f9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:59.893088 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f997e-deb5-42b8-8b6d-75597896fd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:59.893750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f997e-deb5-42b8-8b6d-75597896fd8c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.894038 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f997e-deb5-42b8-8b6d-75597896fd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.894316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f997e-deb5-42b8-8b6d-75597896fd8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.895094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f997e-deb5-42b8-8b6d-75597896fd8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:59.896718 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f997e-deb5-42b8-8b6d-75597896fd8c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:59.907311 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1265/2508] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:35:59.912034 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f997e-deb5-42b8-8b6d-75597896fd8c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.922462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f997e-deb5-42b8-8b6d-75597896fd8c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:35:59.928713 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8560bd5-be49-413a-af66-354a9ef433e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:35:59.929588 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8560bd5-be49-413a-af66-354a9ef433e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:35:59.929875 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8560bd5-be49-413a-af66-354a9ef433e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:35:59.930162 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8560bd5-be49-413a-af66-354a9ef433e6 admin admin] PATH_INFO: `/v3/projects/93898a2c71534bd6ab92c09516512659/users/e544c53462d745bfa62325cfcbe92a8a/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:35:59.930700 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1244/2509] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:35:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:35:59.941275 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32480c-a715-4dc4-b726-864403e1f9d6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.942673 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9e32480c-a715-4dc4-b726-864403e1f9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:35:59.945930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8560bd5-be49-413a-af66-354a9ef433e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93898a2c71534bd6ab92c09516512659, user_id=e544c53462d745bfa62325cfcbe92a8a, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:35:59.948676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8560bd5-be49-413a-af66-354a9ef433e6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:35:59.962557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1266/2510] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:35:59 2022] PUT /identity/v3/projects/93898a2c71534bd6ab92c09516512659/users/e544c53462d745bfa62325cfcbe92a8a/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:35:59.966116 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8560bd5-be49-413a-af66-354a9ef433e6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:35:59.969000 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:00.006845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32480c-a715-4dc4-b726-864403e1f9d6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:00.027998 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8560bd5-be49-413a-af66-354a9ef433e6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:00.033241 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1604d-f17a-41b1-b979-9f8ae68e9621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:00.033991 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1604d-f17a-41b1-b979-9f8ae68e9621 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:00.034283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1604d-f17a-41b1-b979-9f8ae68e9621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:00.034580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1604d-f17a-41b1-b979-9f8ae68e9621 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:00.035298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1604d-f17a-41b1-b979-9f8ae68e9621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:00.036934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1604d-f17a-41b1-b979-9f8ae68e9621 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:00.046768 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1267/2511] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:36:00.053462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1604d-f17a-41b1-b979-9f8ae68e9621 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:00.056529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c740c6c-f16a-4c4b-93b3-969722c1072a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:00.057291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c740c6c-f16a-4c4b-93b3-969722c1072a service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:00.057701 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c740c6c-f16a-4c4b-93b3-969722c1072a service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:00.058073 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c740c6c-f16a-4c4b-93b3-969722c1072a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:00.065488 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1604d-f17a-41b1-b979-9f8ae68e9621 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:00.070740 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84a63a-49a8-4587-9b41-2b2eb4b48d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:00.071893 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84a63a-49a8-4587-9b41-2b2eb4b48d92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:00.072186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84a63a-49a8-4587-9b41-2b2eb4b48d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:00.072468 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84a63a-49a8-4587-9b41-2b2eb4b48d92 admin admin] PATH_INFO: `/v3/projects/93898a2c71534bd6ab92c09516512659/users/e544c53462d745bfa62325cfcbe92a8a/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:00.075149 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84a63a-49a8-4587-9b41-2b2eb4b48d92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93898a2c71534bd6ab92c09516512659, user_id=e544c53462d745bfa62325cfcbe92a8a, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:00.078057 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84a63a-49a8-4587-9b41-2b2eb4b48d92 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:00.096700 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1268/2512] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:00 2022] PUT /identity/v3/projects/93898a2c71534bd6ab92c09516512659/users/e544c53462d745bfa62325cfcbe92a8a/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:00.100483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29323114-0891-48cb-85f8-bb04c5e566a1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:00.100908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29323114-0891-48cb-85f8-bb04c5e566a1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:00.101259 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29323114-0891-48cb-85f8-bb04c5e566a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:00.118722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c740c6c-f16a-4c4b-93b3-969722c1072a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:00.120168 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c740c6c-f16a-4c4b-93b3-969722c1072a service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:00.129777 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1245/2513] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:35:59 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:36:00.132907 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-29323114-0891-48cb-85f8-bb04c5e566a1 None None] MFA Rules not processed for user `e544c53462d745bfa62325cfcbe92a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:00.134152 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-29323114-0891-48cb-85f8-bb04c5e566a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:00.147486 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:00.210238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1269/2514] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:36:00.222345 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c740c6c-f16a-4c4b-93b3-969722c1072a service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:00.224069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8c740c6c-f16a-4c4b-93b3-969722c1072a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:00.235656 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c740c6c-f16a-4c4b-93b3-969722c1072a service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:00.285161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37af3ed-986a-4836-bfb2-f56506fc4ae9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:00.286006 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37af3ed-986a-4836-bfb2-f56506fc4ae9 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:00.286288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37af3ed-986a-4836-bfb2-f56506fc4ae9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:00.286572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37af3ed-986a-4836-bfb2-f56506fc4ae9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:00.296101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37af3ed-986a-4836-bfb2-f56506fc4ae9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:00.297573 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37af3ed-986a-4836-bfb2-f56506fc4ae9 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:00.307836 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1246/2515] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:00.394419 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29323114-0891-48cb-85f8-bb04c5e566a1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:00.405584 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29323114-0891-48cb-85f8-bb04c5e566a1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:00.410230 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75507592-48ee-4fe9-9adf-0a38f45b489d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:00.411044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75507592-48ee-4fe9-9adf-0a38f45b489d service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:00.411425 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75507592-48ee-4fe9-9adf-0a38f45b489d service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:00.412179 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75507592-48ee-4fe9-9adf-0a38f45b489d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:00.483011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75507592-48ee-4fe9-9adf-0a38f45b489d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:00.484634 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75507592-48ee-4fe9-9adf-0a38f45b489d service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:00.494981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1270/2516] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:09.571353 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37af3ed-986a-4836-bfb2-f56506fc4ae9 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:09.607473 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:09.637661 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37af3ed-986a-4836-bfb2-f56506fc4ae9 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:09.642247 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281496a-4d45-4f1b-ab81-59f6f7eb2049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:09.643126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281496a-4d45-4f1b-ab81-59f6f7eb2049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:09.643411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281496a-4d45-4f1b-ab81-59f6f7eb2049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:09.643720 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281496a-4d45-4f1b-ab81-59f6f7eb2049 admin admin] PATH_INFO: `/v3/users/bb5e75c6aae84a8d857a4377c22a1ba1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:09.644911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281496a-4d45-4f1b-ab81-59f6f7eb2049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb5e75c6aae84a8d857a4377c22a1ba1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:09.647089 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281496a-4d45-4f1b-ab81-59f6f7eb2049 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:09.695615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f281496a-4d45-4f1b-ab81-59f6f7eb2049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb5e75c6aae84a8d857a4377c22a1ba1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:09.696271 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f281496a-4d45-4f1b-ab81-59f6f7eb2049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb5e75c6aae84a8d857a4377c22a1ba1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:09.696855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f281496a-4d45-4f1b-ab81-59f6f7eb2049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb5e75c6aae84a8d857a4377c22a1ba1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:09.704445 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f281496a-4d45-4f1b-ab81-59f6f7eb2049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb5e75c6aae84a8d857a4377c22a1ba1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:09.717490 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f281496a-4d45-4f1b-ab81-59f6f7eb2049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb5e75c6aae84a8d857a4377c22a1ba1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:09.730945 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1247/2517] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:36:09 2022] DELETE /identity/v3/users/bb5e75c6aae84a8d857a4377c22a1ba1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:09.735675 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3e87f-620f-40a9-ae5b-102674238434 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:09.736247 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3e87f-620f-40a9-ae5b-102674238434 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:09.736390 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3e87f-620f-40a9-ae5b-102674238434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:09.753915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c0c3e87f-620f-40a9-ae5b-102674238434 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:09.755121 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c0c3e87f-620f-40a9-ae5b-102674238434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:09.814963 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1271/2518] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:09.824683 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281496a-4d45-4f1b-ab81-59f6f7eb2049 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:09.826058 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f281496a-4d45-4f1b-ab81-59f6f7eb2049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:09.884764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281496a-4d45-4f1b-ab81-59f6f7eb2049 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:09.888337 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a4981-a580-4e39-8094-f563c5959d54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:09.889007 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a4981-a580-4e39-8094-f563c5959d54 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:09.889289 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a4981-a580-4e39-8094-f563c5959d54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:09.889564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a4981-a580-4e39-8094-f563c5959d54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:09.899446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a4981-a580-4e39-8094-f563c5959d54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:09.901039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a4981-a580-4e39-8094-f563c5959d54 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:09.912187 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1248/2519] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:10.276117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3e87f-620f-40a9-ae5b-102674238434 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:10.277652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c0c3e87f-620f-40a9-ae5b-102674238434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:10.290482 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3e87f-620f-40a9-ae5b-102674238434 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:10.296050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e67503-e535-44d4-ae10-fa183758c497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:10.297568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e67503-e535-44d4-ae10-fa183758c497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:10.297568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e67503-e535-44d4-ae10-fa183758c497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:10.297919 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e67503-e535-44d4-ae10-fa183758c497 admin admin] PATH_INFO: `/v3/projects/43169a2c7a3b4545b6770fa990652739` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:10.299827 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e67503-e535-44d4-ae10-fa183758c497 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43169a2c7a3b4545b6770fa990652739)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:10.301767 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e67503-e535-44d4-ae10-fa183758c497 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:10.346508 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:10.350500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-24e67503-e535-44d4-ae10-fa183758c497 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43169a2c7a3b4545b6770fa990652739', 'request_id': 'req-24e67503-e535-44d4-ae10-fa183758c497'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:10.361702 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-24e67503-e535-44d4-ae10-fa183758c497 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43169a2c7a3b4545b6770fa990652739', 'request_id': 'req-24e67503-e535-44d4-ae10-fa183758c497'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:10.367873 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-24e67503-e535-44d4-ae10-fa183758c497 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43169a2c7a3b4545b6770fa990652739', 'request_id': 'req-24e67503-e535-44d4-ae10-fa183758c497'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:10.380610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-24e67503-e535-44d4-ae10-fa183758c497 admin admin] The token cache is being invalidate because project 43169a2c7a3b4545b6770fa990652739 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:36:10.380930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-24e67503-e535-44d4-ae10-fa183758c497 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:10.381555 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-24e67503-e535-44d4-ae10-fa183758c497 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:10.383301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1272/2520] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:36:10 2022] DELETE /identity/v3/projects/43169a2c7a3b4545b6770fa990652739 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:10.388009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0f383-7c4c-41cc-a581-39dd3546cf68 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:10.388311 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0f383-7c4c-41cc-a581-39dd3546cf68 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:10.388624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0f383-7c4c-41cc-a581-39dd3546cf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:10.408084 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1ff0f383-7c4c-41cc-a581-39dd3546cf68 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:10.409351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1ff0f383-7c4c-41cc-a581-39dd3546cf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:10.456356 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:10.470572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1249/2521] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:10.478703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c9650-3506-44c2-8b5c-137170a02237 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:10.478997 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c9650-3506-44c2-8b5c-137170a02237 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:10.479278 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c9650-3506-44c2-8b5c-137170a02237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:10.497957 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a36c9650-3506-44c2-8b5c-137170a02237 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:10.499160 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a36c9650-3506-44c2-8b5c-137170a02237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:10.507734 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1273/2522] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:10.511912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0f383-7c4c-41cc-a581-39dd3546cf68 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:10.523695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0f383-7c4c-41cc-a581-39dd3546cf68 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:10.529273 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245804f-3810-47f9-ba3d-5ef20513728e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:10.529939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245804f-3810-47f9-ba3d-5ef20513728e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:10.530227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245804f-3810-47f9-ba3d-5ef20513728e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:10.530742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245804f-3810-47f9-ba3d-5ef20513728e admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:10.531238 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245804f-3810-47f9-ba3d-5ef20513728e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:36:10.531547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245804f-3810-47f9-ba3d-5ef20513728e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:10.533195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245804f-3810-47f9-ba3d-5ef20513728e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:10.552673 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1250/2523] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:36:10.556049 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c9650-3506-44c2-8b5c-137170a02237 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:10.566573 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c9650-3506-44c2-8b5c-137170a02237 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:10.571515 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cdd7a-b2b0-4d61-b307-a6347d8615c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:10.572250 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cdd7a-b2b0-4d61-b307-a6347d8615c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:10.572639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cdd7a-b2b0-4d61-b307-a6347d8615c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:10.572986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cdd7a-b2b0-4d61-b307-a6347d8615c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:10.573766 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cdd7a-b2b0-4d61-b307-a6347d8615c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:10.575911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cdd7a-b2b0-4d61-b307-a6347d8615c4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:10.603710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1274/2524] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:36:10 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:10.607875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245804f-3810-47f9-ba3d-5ef20513728e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:10.619027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245804f-3810-47f9-ba3d-5ef20513728e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:10.680125 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efad82-5300-4b5f-8d2e-daf79bc5c654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:10.680993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efad82-5300-4b5f-8d2e-daf79bc5c654 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:10.681414 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efad82-5300-4b5f-8d2e-daf79bc5c654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:10.681696 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efad82-5300-4b5f-8d2e-daf79bc5c654 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:10.682452 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efad82-5300-4b5f-8d2e-daf79bc5c654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:10.684487 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efad82-5300-4b5f-8d2e-daf79bc5c654 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:10.707485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1251/2525] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:36:10 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:10.711952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cdd7a-b2b0-4d61-b307-a6347d8615c4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:10.723304 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cdd7a-b2b0-4d61-b307-a6347d8615c4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:10.728321 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b556e7-11ff-4a84-a2c5-6ca7b3dc61f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:10.729096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b556e7-11ff-4a84-a2c5-6ca7b3dc61f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:10.729377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b556e7-11ff-4a84-a2c5-6ca7b3dc61f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:10.729649 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b556e7-11ff-4a84-a2c5-6ca7b3dc61f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:10.730511 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b556e7-11ff-4a84-a2c5-6ca7b3dc61f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:10.731960 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b556e7-11ff-4a84-a2c5-6ca7b3dc61f5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:10.742761 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1275/2526] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:10.747026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efad82-5300-4b5f-8d2e-daf79bc5c654 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:10.758290 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efad82-5300-4b5f-8d2e-daf79bc5c654 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:10.763446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c786a3-2679-4ce8-945a-4d9803ce3ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:10.764682 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c786a3-2679-4ce8-945a-4d9803ce3ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:10.765003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c786a3-2679-4ce8-945a-4d9803ce3ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:10.765303 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c786a3-2679-4ce8-945a-4d9803ce3ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:10.766038 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c786a3-2679-4ce8-945a-4d9803ce3ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:10.768175 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c786a3-2679-4ce8-945a-4d9803ce3ee8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:10.782373 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1252/2527] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:10.783680 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b556e7-11ff-4a84-a2c5-6ca7b3dc61f5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:10.793903 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b556e7-11ff-4a84-a2c5-6ca7b3dc61f5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:10.799041 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d340829-5820-4802-8b14-727dfa4f2a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:10.799915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d340829-5820-4802-8b14-727dfa4f2a91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:10.800197 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d340829-5820-4802-8b14-727dfa4f2a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:10.800475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d340829-5820-4802-8b14-727dfa4f2a91 admin admin] PATH_INFO: `/v3/projects/904792185582403a89f6e159239a0990/users/894be9d95cdd45bc842e6ee5a7f7cb97/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:10.816120 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d340829-5820-4802-8b14-727dfa4f2a91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=904792185582403a89f6e159239a0990, user_id=894be9d95cdd45bc842e6ee5a7f7cb97, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:10.819061 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d340829-5820-4802-8b14-727dfa4f2a91 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:10.830986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1276/2528] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:10 2022] PUT /identity/v3/projects/904792185582403a89f6e159239a0990/users/894be9d95cdd45bc842e6ee5a7f7cb97/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:10.843850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827d8fe-2d57-4d6f-bd1f-51255e715a6c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:10.844145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827d8fe-2d57-4d6f-bd1f-51255e715a6c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:10.844441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827d8fe-2d57-4d6f-bd1f-51255e715a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:10.871345 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4827d8fe-2d57-4d6f-bd1f-51255e715a6c None None] MFA Rules not processed for user `894be9d95cdd45bc842e6ee5a7f7cb97`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:10.872565 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4827d8fe-2d57-4d6f-bd1f-51255e715a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:10.937274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1253/2529] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:10.942246 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d340829-5820-4802-8b14-727dfa4f2a91 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:11.004854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d340829-5820-4802-8b14-727dfa4f2a91 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:11.009569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf17834-b38f-4e7f-a01b-e05f728690c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:11.010222 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf17834-b38f-4e7f-a01b-e05f728690c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:11.010510 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf17834-b38f-4e7f-a01b-e05f728690c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:11.010785 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf17834-b38f-4e7f-a01b-e05f728690c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:11.011466 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf17834-b38f-4e7f-a01b-e05f728690c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:11.013424 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf17834-b38f-4e7f-a01b-e05f728690c3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:11.039061 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1277/2530] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:36:10 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:11.043151 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827d8fe-2d57-4d6f-bd1f-51255e715a6c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:11.054044 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827d8fe-2d57-4d6f-bd1f-51255e715a6c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:11.096564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:11.109655 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629194a-26ee-4f83-90c2-2053d2e7c61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:11.110340 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629194a-26ee-4f83-90c2-2053d2e7c61b admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:11.110652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629194a-26ee-4f83-90c2-2053d2e7c61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:11.111510 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629194a-26ee-4f83-90c2-2053d2e7c61b admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:11.112211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629194a-26ee-4f83-90c2-2053d2e7c61b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:11.114176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629194a-26ee-4f83-90c2-2053d2e7c61b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:11.133069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1254/2531] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:36:11 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:36:11.137216 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf17834-b38f-4e7f-a01b-e05f728690c3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:11.147406 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf17834-b38f-4e7f-a01b-e05f728690c3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:11.152212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641aebe9-c9d4-4750-a110-3fdc99390bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:11.152910 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641aebe9-c9d4-4750-a110-3fdc99390bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:11.153199 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641aebe9-c9d4-4750-a110-3fdc99390bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:11.153477 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641aebe9-c9d4-4750-a110-3fdc99390bd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:11.154208 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641aebe9-c9d4-4750-a110-3fdc99390bd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:11.155806 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641aebe9-c9d4-4750-a110-3fdc99390bd1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:11.166985 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1278/2532] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:11.171104 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629194a-26ee-4f83-90c2-2053d2e7c61b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:11.181055 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629194a-26ee-4f83-90c2-2053d2e7c61b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:11.185592 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fbc8a-b0c7-4cd0-a493-ca63269f0556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:11.186483 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fbc8a-b0c7-4cd0-a493-ca63269f0556 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:11.186745 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fbc8a-b0c7-4cd0-a493-ca63269f0556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:11.187031 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fbc8a-b0c7-4cd0-a493-ca63269f0556 admin admin] PATH_INFO: `/v3/projects/7362380f708b412682fbe2a35c6f4616/users/0c215ceacc9c4ae1974e5650c86dc624/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:11.202100 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fbc8a-b0c7-4cd0-a493-ca63269f0556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7362380f708b412682fbe2a35c6f4616, user_id=0c215ceacc9c4ae1974e5650c86dc624, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:11.205416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fbc8a-b0c7-4cd0-a493-ca63269f0556 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:11.217894 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1255/2533] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:11 2022] PUT /identity/v3/projects/7362380f708b412682fbe2a35c6f4616/users/0c215ceacc9c4ae1974e5650c86dc624/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:11.221854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641aebe9-c9d4-4750-a110-3fdc99390bd1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:11.232223 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641aebe9-c9d4-4750-a110-3fdc99390bd1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:11.274761 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:11.296100 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e198e-c21b-4d18-9ece-fd57cc22be3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:11.296815 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e198e-c21b-4d18-9ece-fd57cc22be3e admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:11.297126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e198e-c21b-4d18-9ece-fd57cc22be3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:11.297407 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e198e-c21b-4d18-9ece-fd57cc22be3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:11.298138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e198e-c21b-4d18-9ece-fd57cc22be3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:11.300250 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e198e-c21b-4d18-9ece-fd57cc22be3e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:11.310614 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1279/2534] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:11.315394 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fbc8a-b0c7-4cd0-a493-ca63269f0556 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:11.325747 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fbc8a-b0c7-4cd0-a493-ca63269f0556 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:11.330344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401c4b1-b1db-42a4-ba69-bb17f4ab3ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:11.331260 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401c4b1-b1db-42a4-ba69-bb17f4ab3ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:11.331550 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401c4b1-b1db-42a4-ba69-bb17f4ab3ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:11.331822 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401c4b1-b1db-42a4-ba69-bb17f4ab3ee1 admin admin] PATH_INFO: `/v3/projects/7362380f708b412682fbe2a35c6f4616/users/0c215ceacc9c4ae1974e5650c86dc624/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:11.334005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401c4b1-b1db-42a4-ba69-bb17f4ab3ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7362380f708b412682fbe2a35c6f4616, user_id=0c215ceacc9c4ae1974e5650c86dc624, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:11.337713 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401c4b1-b1db-42a4-ba69-bb17f4ab3ee1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:11.356156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1256/2535] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:11 2022] PUT /identity/v3/projects/7362380f708b412682fbe2a35c6f4616/users/0c215ceacc9c4ae1974e5650c86dc624/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:11.367868 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7adef-dff7-4db2-986e-c339a718ed39 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:11.368607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7adef-dff7-4db2-986e-c339a718ed39 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:11.369237 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7adef-dff7-4db2-986e-c339a718ed39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:11.398899 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7ff7adef-dff7-4db2-986e-c339a718ed39 None None] MFA Rules not processed for user `0c215ceacc9c4ae1974e5650c86dc624`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:11.400125 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7ff7adef-dff7-4db2-986e-c339a718ed39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:11.467658 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1280/2536] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:11.479836 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401c4b1-b1db-42a4-ba69-bb17f4ab3ee1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:11.481625 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a401c4b1-b1db-42a4-ba69-bb17f4ab3ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:11.537560 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401c4b1-b1db-42a4-ba69-bb17f4ab3ee1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:11.541347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e90cb8-1bde-4ce8-811d-d5e478a96e0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:11.541930 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e90cb8-1bde-4ce8-811d-d5e478a96e0a service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:11.542214 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e90cb8-1bde-4ce8-811d-d5e478a96e0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:11.542544 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e90cb8-1bde-4ce8-811d-d5e478a96e0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:11.551955 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e90cb8-1bde-4ce8-811d-d5e478a96e0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:11.553411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e90cb8-1bde-4ce8-811d-d5e478a96e0a service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:11.562691 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1257/2537] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:11.878809 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7adef-dff7-4db2-986e-c339a718ed39 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:11.888856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7adef-dff7-4db2-986e-c339a718ed39 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:11.894068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c18943-5dee-47fc-8ea2-6757d7e1e837 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c215ceacc9c4ae1974e5650c86dc624', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7362380f708b412682fbe2a35c6f4616', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:11.894855 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c18943-5dee-47fc-8ea2-6757d7e1e837 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:11.895153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c18943-5dee-47fc-8ea2-6757d7e1e837 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:11.895436 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c18943-5dee-47fc-8ea2-6757d7e1e837 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:11.896175 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c18943-5dee-47fc-8ea2-6757d7e1e837 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:36:11.896488 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c18943-5dee-47fc-8ea2-6757d7e1e837 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:11.898153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c18943-5dee-47fc-8ea2-6757d7e1e837 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:11.921000 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1281/2538] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:36:11.925644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e90cb8-1bde-4ce8-811d-d5e478a96e0a service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:11.935736 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e90cb8-1bde-4ce8-811d-d5e478a96e0a service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:11.941829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddddbe-cfe4-4ea2-8ffb-c2cd4ae4a9c3 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c215ceacc9c4ae1974e5650c86dc624', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7362380f708b412682fbe2a35c6f4616', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:11.941829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddddbe-cfe4-4ea2-8ffb-c2cd4ae4a9c3 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:11.942092 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddddbe-cfe4-4ea2-8ffb-c2cd4ae4a9c3 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:11.942175 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddddbe-cfe4-4ea2-8ffb-c2cd4ae4a9c3 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:11.942987 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ddddbe-cfe4-4ea2-8ffb-c2cd4ae4a9c3 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:11.944876 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ddddbe-cfe4-4ea2-8ffb-c2cd4ae4a9c3 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:11.978333 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1258/2539] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:36:11 2022] POST /identity/v3/projects => generated 354 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:11.979978 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c18943-5dee-47fc-8ea2-6757d7e1e837 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:11.991543 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c18943-5dee-47fc-8ea2-6757d7e1e837 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:12.087537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd74cd-3bc0-4e71-b761-7489f66ec503 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c215ceacc9c4ae1974e5650c86dc624', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7362380f708b412682fbe2a35c6f4616', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:12.088416 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd74cd-3bc0-4e71-b761-7489f66ec503 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:12.088745 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd74cd-3bc0-4e71-b761-7489f66ec503 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:12.089027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd74cd-3bc0-4e71-b761-7489f66ec503 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:12.090398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd74cd-3bc0-4e71-b761-7489f66ec503 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:36:12.090781 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd74cd-3bc0-4e71-b761-7489f66ec503 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:12.092817 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd74cd-3bc0-4e71-b761-7489f66ec503 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:12.107537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1282/2540] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:36:12.147128 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddddbe-cfe4-4ea2-8ffb-c2cd4ae4a9c3 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:12.160317 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddddbe-cfe4-4ea2-8ffb-c2cd4ae4a9c3 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:12.165115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d98b5-d55c-4309-b59b-61ee957eae10 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c215ceacc9c4ae1974e5650c86dc624', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7362380f708b412682fbe2a35c6f4616', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:12.166000 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d98b5-d55c-4309-b59b-61ee957eae10 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:12.166291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d98b5-d55c-4309-b59b-61ee957eae10 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:12.166583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d98b5-d55c-4309-b59b-61ee957eae10 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] PATH_INFO: `/v3/projects/2f29f3952c844a43abb65405b0f3909a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:12.167786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d98b5-d55c-4309-b59b-61ee957eae10 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2f29f3952c844a43abb65405b0f3909a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:12.169895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d98b5-d55c-4309-b59b-61ee957eae10 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:12.216866 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b98d98b5-d55c-4309-b59b-61ee957eae10 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f29f3952c844a43abb65405b0f3909a', 'request_id': 'req-b98d98b5-d55c-4309-b59b-61ee957eae10'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:12.223070 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b98d98b5-d55c-4309-b59b-61ee957eae10 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f29f3952c844a43abb65405b0f3909a', 'request_id': 'req-b98d98b5-d55c-4309-b59b-61ee957eae10'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:12.231039 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b98d98b5-d55c-4309-b59b-61ee957eae10 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f29f3952c844a43abb65405b0f3909a', 'request_id': 'req-b98d98b5-d55c-4309-b59b-61ee957eae10'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:12.243278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b98d98b5-d55c-4309-b59b-61ee957eae10 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] The token cache is being invalidate because project 2f29f3952c844a43abb65405b0f3909a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:36:12.243548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b98d98b5-d55c-4309-b59b-61ee957eae10 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:12.244242 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b98d98b5-d55c-4309-b59b-61ee957eae10 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:12.245864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1259/2541] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:36:12 2022] DELETE /identity/v3/projects/2f29f3952c844a43abb65405b0f3909a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:12.443686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd74cd-3bc0-4e71-b761-7489f66ec503 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:12.445882 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-49fd74cd-3bc0-4e71-b761-7489f66ec503 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:12.495520 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:12.515606 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd74cd-3bc0-4e71-b761-7489f66ec503 tempest-VolumeQuotaClassesTest-477746572 tempest-VolumeQuotaClassesTest-477746572-project-admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:12.575297 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b686d9-ed69-4ada-a6af-22c59a79d7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:12.576204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b686d9-ed69-4ada-a6af-22c59a79d7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:12.576488 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b686d9-ed69-4ada-a6af-22c59a79d7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:12.576798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b686d9-ed69-4ada-a6af-22c59a79d7c8 admin admin] PATH_INFO: `/v3/users/894be9d95cdd45bc842e6ee5a7f7cb97` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:12.578046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b686d9-ed69-4ada-a6af-22c59a79d7c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=894be9d95cdd45bc842e6ee5a7f7cb97)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:12.580917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b686d9-ed69-4ada-a6af-22c59a79d7c8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:12.631194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-16b686d9-ed69-4ada-a6af-22c59a79d7c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '894be9d95cdd45bc842e6ee5a7f7cb97'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:12.646395 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-16b686d9-ed69-4ada-a6af-22c59a79d7c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '894be9d95cdd45bc842e6ee5a7f7cb97'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:12.647151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-16b686d9-ed69-4ada-a6af-22c59a79d7c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '894be9d95cdd45bc842e6ee5a7f7cb97'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:12.647754 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-16b686d9-ed69-4ada-a6af-22c59a79d7c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '894be9d95cdd45bc842e6ee5a7f7cb97'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:12.657209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-16b686d9-ed69-4ada-a6af-22c59a79d7c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '894be9d95cdd45bc842e6ee5a7f7cb97'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:12.670050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1283/2542] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:36:12 2022] DELETE /identity/v3/users/894be9d95cdd45bc842e6ee5a7f7cb97 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:12.674558 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3796e7-04c2-4db0-86da-c0eb807042f2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:12.674920 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3796e7-04c2-4db0-86da-c0eb807042f2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:12.675207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3796e7-04c2-4db0-86da-c0eb807042f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:12.681932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:12.693877 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bb3796e7-04c2-4db0-86da-c0eb807042f2 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:12.695069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bb3796e7-04c2-4db0-86da-c0eb807042f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:12.753804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1260/2543] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:36:12.763512 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b686d9-ed69-4ada-a6af-22c59a79d7c8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:12.764961 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-16b686d9-ed69-4ada-a6af-22c59a79d7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:12.821155 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b686d9-ed69-4ada-a6af-22c59a79d7c8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:12.824964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b163d71-3d39-4b84-93d9-e145f54e8f3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:12.825616 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b163d71-3d39-4b84-93d9-e145f54e8f3d service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:12.825879 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b163d71-3d39-4b84-93d9-e145f54e8f3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:12.826154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b163d71-3d39-4b84-93d9-e145f54e8f3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:12.835172 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b163d71-3d39-4b84-93d9-e145f54e8f3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:12.836631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b163d71-3d39-4b84-93d9-e145f54e8f3d service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:12.846209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1284/2544] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:13.112629 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3796e7-04c2-4db0-86da-c0eb807042f2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:13.114070 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bb3796e7-04c2-4db0-86da-c0eb807042f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:13.128183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3796e7-04c2-4db0-86da-c0eb807042f2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:13.133288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0305a0-87d2-462e-89bb-800785268824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:13.134940 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0305a0-87d2-462e-89bb-800785268824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:13.135229 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0305a0-87d2-462e-89bb-800785268824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:13.135511 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0305a0-87d2-462e-89bb-800785268824 admin admin] PATH_INFO: `/v3/projects/904792185582403a89f6e159239a0990` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:13.144884 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0305a0-87d2-462e-89bb-800785268824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=904792185582403a89f6e159239a0990)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:13.146951 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0305a0-87d2-462e-89bb-800785268824 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:13.213111 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0e0305a0-87d2-462e-89bb-800785268824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '904792185582403a89f6e159239a0990', 'request_id': 'req-0e0305a0-87d2-462e-89bb-800785268824'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:13.220997 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0e0305a0-87d2-462e-89bb-800785268824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '904792185582403a89f6e159239a0990', 'request_id': 'req-0e0305a0-87d2-462e-89bb-800785268824'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:13.228216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0e0305a0-87d2-462e-89bb-800785268824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '904792185582403a89f6e159239a0990', 'request_id': 'req-0e0305a0-87d2-462e-89bb-800785268824'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:13.242081 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0e0305a0-87d2-462e-89bb-800785268824 admin admin] The token cache is being invalidate because project 904792185582403a89f6e159239a0990 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:36:13.242369 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0e0305a0-87d2-462e-89bb-800785268824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:13.242968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0e0305a0-87d2-462e-89bb-800785268824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:13.244471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1261/2545] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:36:13 2022] DELETE /identity/v3/projects/904792185582403a89f6e159239a0990 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:13.248476 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b163d71-3d39-4b84-93d9-e145f54e8f3d service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:13.260588 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b163d71-3d39-4b84-93d9-e145f54e8f3d service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:13.270341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:13.324999 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054351f-3001-401b-afd7-360a89068b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:13.325882 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054351f-3001-401b-afd7-360a89068b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:13.326170 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054351f-3001-401b-afd7-360a89068b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:13.326459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054351f-3001-401b-afd7-360a89068b5c admin admin] PATH_INFO: `/v3/users/0c215ceacc9c4ae1974e5650c86dc624` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:13.327933 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054351f-3001-401b-afd7-360a89068b5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c215ceacc9c4ae1974e5650c86dc624)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:13.330922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054351f-3001-401b-afd7-360a89068b5c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:13.389608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3054351f-3001-401b-afd7-360a89068b5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c215ceacc9c4ae1974e5650c86dc624'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:13.411688 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3054351f-3001-401b-afd7-360a89068b5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c215ceacc9c4ae1974e5650c86dc624'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:13.412623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3054351f-3001-401b-afd7-360a89068b5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c215ceacc9c4ae1974e5650c86dc624'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:13.413235 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3054351f-3001-401b-afd7-360a89068b5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c215ceacc9c4ae1974e5650c86dc624'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:13.421234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3054351f-3001-401b-afd7-360a89068b5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c215ceacc9c4ae1974e5650c86dc624'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:13.433759 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1285/2546] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:36:13 2022] DELETE /identity/v3/users/0c215ceacc9c4ae1974e5650c86dc624 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:13.703002 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0305a0-87d2-462e-89bb-800785268824 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:13.705745 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0e0305a0-87d2-462e-89bb-800785268824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:13.721240 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0305a0-87d2-462e-89bb-800785268824 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:13.784143 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14034d-d3b3-46c5-ae25-4686589fbad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:13.785144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14034d-d3b3-46c5-ae25-4686589fbad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:13.785429 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14034d-d3b3-46c5-ae25-4686589fbad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:13.785704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14034d-d3b3-46c5-ae25-4686589fbad1 admin admin] PATH_INFO: `/v3/projects/7362380f708b412682fbe2a35c6f4616` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:13.786951 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14034d-d3b3-46c5-ae25-4686589fbad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7362380f708b412682fbe2a35c6f4616)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:13.788983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14034d-d3b3-46c5-ae25-4686589fbad1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:13.818011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addaf62-fb04-4b4c-b4af-b2f1e5a1fdcb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:13.818378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addaf62-fb04-4b4c-b4af-b2f1e5a1fdcb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:13.818926 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addaf62-fb04-4b4c-b4af-b2f1e5a1fdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:13.836208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:13.838174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8addaf62-fb04-4b4c-b4af-b2f1e5a1fdcb None None] MFA Rules not processed for user `a09029144c454b099bea86aaa5229ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:13.839484 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8addaf62-fb04-4b4c-b4af-b2f1e5a1fdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:13.856850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ef14034d-d3b3-46c5-ae25-4686589fbad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7362380f708b412682fbe2a35c6f4616', 'request_id': 'req-ef14034d-d3b3-46c5-ae25-4686589fbad1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:13.862622 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ef14034d-d3b3-46c5-ae25-4686589fbad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7362380f708b412682fbe2a35c6f4616', 'request_id': 'req-ef14034d-d3b3-46c5-ae25-4686589fbad1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:13.868868 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ef14034d-d3b3-46c5-ae25-4686589fbad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7362380f708b412682fbe2a35c6f4616', 'request_id': 'req-ef14034d-d3b3-46c5-ae25-4686589fbad1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:13.881956 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ef14034d-d3b3-46c5-ae25-4686589fbad1 admin admin] The token cache is being invalidate because project 7362380f708b412682fbe2a35c6f4616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:36:13.882246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ef14034d-d3b3-46c5-ae25-4686589fbad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:13.882871 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ef14034d-d3b3-46c5-ae25-4686589fbad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:13.884415 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1262/2547] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:36:13 2022] DELETE /identity/v3/projects/7362380f708b412682fbe2a35c6f4616 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:13.889510 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364e2e0-eb17-4a51-9b7d-39e8ccc59449 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:13.889799 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364e2e0-eb17-4a51-9b7d-39e8ccc59449 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:13.890079 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364e2e0-eb17-4a51-9b7d-39e8ccc59449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:13.898793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1286/2548] 10.222.0.33 () {58 vars in 967 bytes} [Mon Jul 18 10:36:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:36:13.909206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6364e2e0-eb17-4a51-9b7d-39e8ccc59449 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:13.910429 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6364e2e0-eb17-4a51-9b7d-39e8ccc59449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:13.911107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8addaf62-fb04-4b4c-b4af-b2f1e5a1fdcb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:13.912437 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8addaf62-fb04-4b4c-b4af-b2f1e5a1fdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:13.954306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:13.971714 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1263/2549] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:13.979677 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4946b-217a-4c08-b669-fb0729e9895a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:13.979971 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4946b-217a-4c08-b669-fb0729e9895a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:13.980254 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e4946b-217a-4c08-b669-fb0729e9895a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:13.985084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8addaf62-fb04-4b4c-b4af-b2f1e5a1fdcb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:13.990300 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627db54-6bd6-4952-aa9c-25f8a720eae6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6168873f2846c8af73b903bba17eb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:13.990980 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627db54-6bd6-4952-aa9c-25f8a720eae6 service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:13.991271 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627db54-6bd6-4952-aa9c-25f8a720eae6 service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:13.991552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627db54-6bd6-4952-aa9c-25f8a720eae6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:13.997811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c2e4946b-217a-4c08-b669-fb0729e9895a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:13.999008 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c2e4946b-217a-4c08-b669-fb0729e9895a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:14.000056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627db54-6bd6-4952-aa9c-25f8a720eae6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:14.002263 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627db54-6bd6-4952-aa9c-25f8a720eae6 service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:14.007651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1264/2550] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:36:14.011218 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1287/2551] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 18 10:36:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 10:36:14.011733 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e4946b-217a-4c08-b669-fb0729e9895a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:14.021989 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e4946b-217a-4c08-b669-fb0729e9895a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:14.026787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e72627-21aa-4d93-b146-f92c7a884dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:14.027443 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e72627-21aa-4d93-b146-f92c7a884dce admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:14.027752 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e72627-21aa-4d93-b146-f92c7a884dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:14.028027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e72627-21aa-4d93-b146-f92c7a884dce admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:14.028813 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e72627-21aa-4d93-b146-f92c7a884dce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:36:14.029117 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e72627-21aa-4d93-b146-f92c7a884dce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:14.030744 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e72627-21aa-4d93-b146-f92c7a884dce admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:14.043142 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1265/2552] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:36:14.051443 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627db54-6bd6-4952-aa9c-25f8a720eae6 service swift] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:14.065406 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627db54-6bd6-4952-aa9c-25f8a720eae6 service swift] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:14.070149 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0a9de-58f0-42c8-85fe-54194710e737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:14.070810 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0a9de-58f0-42c8-85fe-54194710e737 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:14.071099 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0a9de-58f0-42c8-85fe-54194710e737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:14.071412 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0a9de-58f0-42c8-85fe-54194710e737 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:14.072112 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0a9de-58f0-42c8-85fe-54194710e737 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:14.074268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0a9de-58f0-42c8-85fe-54194710e737 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:14.102030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1288/2553] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:36:14 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:36:14.106241 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e72627-21aa-4d93-b146-f92c7a884dce admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:14.116315 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e72627-21aa-4d93-b146-f92c7a884dce admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:14.179224 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55e46c-f8ee-4f05-93df-03b85253323c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:14.179973 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55e46c-f8ee-4f05-93df-03b85253323c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:14.180259 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55e46c-f8ee-4f05-93df-03b85253323c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:14.180568 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55e46c-f8ee-4f05-93df-03b85253323c admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:14.181275 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55e46c-f8ee-4f05-93df-03b85253323c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:14.183327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55e46c-f8ee-4f05-93df-03b85253323c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:14.205264 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1266/2554] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:36:14 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:14.209045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0a9de-58f0-42c8-85fe-54194710e737 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:14.219675 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0a9de-58f0-42c8-85fe-54194710e737 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:14.224672 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad18e60-e467-48a4-8d88-b73cf3feb663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:14.225343 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad18e60-e467-48a4-8d88-b73cf3feb663 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:14.225631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad18e60-e467-48a4-8d88-b73cf3feb663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:14.225911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad18e60-e467-48a4-8d88-b73cf3feb663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:14.226912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad18e60-e467-48a4-8d88-b73cf3feb663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:14.228525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad18e60-e467-48a4-8d88-b73cf3feb663 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:14.240161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1289/2555] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:14.244221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55e46c-f8ee-4f05-93df-03b85253323c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:14.254827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55e46c-f8ee-4f05-93df-03b85253323c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:14.259496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e6550-6d7e-4692-b5fa-5b393028e034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:14.260155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e6550-6d7e-4692-b5fa-5b393028e034 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:14.260441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e6550-6d7e-4692-b5fa-5b393028e034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:14.260744 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e6550-6d7e-4692-b5fa-5b393028e034 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:14.261474 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e6550-6d7e-4692-b5fa-5b393028e034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:14.263071 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e6550-6d7e-4692-b5fa-5b393028e034 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:14.273805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1267/2556] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:14.277929 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad18e60-e467-48a4-8d88-b73cf3feb663 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:14.287779 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad18e60-e467-48a4-8d88-b73cf3feb663 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:14.292599 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0764f-5b63-4aae-818e-0bf5a22edf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:14.293512 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0764f-5b63-4aae-818e-0bf5a22edf9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:14.293821 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0764f-5b63-4aae-818e-0bf5a22edf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:14.294077 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0764f-5b63-4aae-818e-0bf5a22edf9f admin admin] PATH_INFO: `/v3/projects/e54589f3a8644fdca82daa89ae7d9fa5/users/9e9543d4ab044991af1e0d5ef4ac1546/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:14.309859 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0764f-5b63-4aae-818e-0bf5a22edf9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e54589f3a8644fdca82daa89ae7d9fa5, user_id=9e9543d4ab044991af1e0d5ef4ac1546, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:14.312623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0764f-5b63-4aae-818e-0bf5a22edf9f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:14.326057 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1290/2557] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:14 2022] PUT /identity/v3/projects/e54589f3a8644fdca82daa89ae7d9fa5/users/9e9543d4ab044991af1e0d5ef4ac1546/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:14.338993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3b931-95c7-426d-9d61-9a3573ef34c5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:14.339291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3b931-95c7-426d-9d61-9a3573ef34c5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:14.339576 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3b931-95c7-426d-9d61-9a3573ef34c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:14.366556 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c7d3b931-95c7-426d-9d61-9a3573ef34c5 None None] MFA Rules not processed for user `9e9543d4ab044991af1e0d5ef4ac1546`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:14.367782 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c7d3b931-95c7-426d-9d61-9a3573ef34c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:14.373038 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:14.437463 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1268/2558] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:14.442164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0764f-5b63-4aae-818e-0bf5a22edf9f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:14.508912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0764f-5b63-4aae-818e-0bf5a22edf9f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:14.519835 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09640b69-89bf-43f2-bba0-b137e9a23199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:14.520760 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09640b69-89bf-43f2-bba0-b137e9a23199 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:14.520970 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09640b69-89bf-43f2-bba0-b137e9a23199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:14.521213 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09640b69-89bf-43f2-bba0-b137e9a23199 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:14.521964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09640b69-89bf-43f2-bba0-b137e9a23199 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:14.523967 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09640b69-89bf-43f2-bba0-b137e9a23199 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:14.556551 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1291/2559] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:36:14 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:14.558485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3b931-95c7-426d-9d61-9a3573ef34c5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:14.570717 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3b931-95c7-426d-9d61-9a3573ef34c5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:14.637192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885c00d-3db0-411d-b2fb-c64635c4fe96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:14.637885 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885c00d-3db0-411d-b2fb-c64635c4fe96 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:14.638166 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885c00d-3db0-411d-b2fb-c64635c4fe96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:14.638571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885c00d-3db0-411d-b2fb-c64635c4fe96 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:14.639131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885c00d-3db0-411d-b2fb-c64635c4fe96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:14.641328 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885c00d-3db0-411d-b2fb-c64635c4fe96 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:14.661632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1269/2560] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:36:14 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:14.666032 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09640b69-89bf-43f2-bba0-b137e9a23199 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:14.676726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09640b69-89bf-43f2-bba0-b137e9a23199 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:14.681519 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f188a00-ea00-4c7e-9329-0b2e9741566c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:14.682180 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f188a00-ea00-4c7e-9329-0b2e9741566c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:14.682475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f188a00-ea00-4c7e-9329-0b2e9741566c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:14.682758 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f188a00-ea00-4c7e-9329-0b2e9741566c admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:14.683524 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f188a00-ea00-4c7e-9329-0b2e9741566c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:14.685292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f188a00-ea00-4c7e-9329-0b2e9741566c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:14.705167 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1292/2561] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:14.709993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885c00d-3db0-411d-b2fb-c64635c4fe96 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:14.721385 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885c00d-3db0-411d-b2fb-c64635c4fe96 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:14.726636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e04c2-840b-49b2-8612-0c344cc2feb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:14.727525 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e04c2-840b-49b2-8612-0c344cc2feb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:14.727811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e04c2-840b-49b2-8612-0c344cc2feb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:14.728089 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e04c2-840b-49b2-8612-0c344cc2feb4 admin admin] PATH_INFO: `/v3/projects/b38c47d257974538b838e5209ac9da4d/users/fa1f21a0d6e342deaa286a8e2bb9ee61/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:14.747105 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e04c2-840b-49b2-8612-0c344cc2feb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38c47d257974538b838e5209ac9da4d, user_id=fa1f21a0d6e342deaa286a8e2bb9ee61, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:14.749898 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e04c2-840b-49b2-8612-0c344cc2feb4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:14.770948 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1270/2562] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:14 2022] PUT /identity/v3/projects/b38c47d257974538b838e5209ac9da4d/users/fa1f21a0d6e342deaa286a8e2bb9ee61/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:14.771835 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f188a00-ea00-4c7e-9329-0b2e9741566c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:14.784145 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f188a00-ea00-4c7e-9329-0b2e9741566c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:14.831246 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:14.848577 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393737a-82b2-4336-b10f-c410fea0458f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:14.849242 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393737a-82b2-4336-b10f-c410fea0458f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:14.849528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393737a-82b2-4336-b10f-c410fea0458f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:14.849953 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393737a-82b2-4336-b10f-c410fea0458f admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:14.850646 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393737a-82b2-4336-b10f-c410fea0458f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:14.852549 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393737a-82b2-4336-b10f-c410fea0458f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:14.865727 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1293/2563] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:36:14.866895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e04c2-840b-49b2-8612-0c344cc2feb4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:14.878069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e04c2-840b-49b2-8612-0c344cc2feb4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:14.882896 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730c2fa-99ce-4f70-abe1-e7ca714a44c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:14.883858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730c2fa-99ce-4f70-abe1-e7ca714a44c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:14.884200 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730c2fa-99ce-4f70-abe1-e7ca714a44c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:14.884564 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730c2fa-99ce-4f70-abe1-e7ca714a44c9 admin admin] PATH_INFO: `/v3/projects/b38c47d257974538b838e5209ac9da4d/users/fa1f21a0d6e342deaa286a8e2bb9ee61/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:14.886754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c730c2fa-99ce-4f70-abe1-e7ca714a44c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38c47d257974538b838e5209ac9da4d, user_id=fa1f21a0d6e342deaa286a8e2bb9ee61, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:14.889593 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c730c2fa-99ce-4f70-abe1-e7ca714a44c9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:14.900592 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1271/2564] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:14 2022] PUT /identity/v3/projects/b38c47d257974538b838e5209ac9da4d/users/fa1f21a0d6e342deaa286a8e2bb9ee61/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:14.914087 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c4b64-5806-493e-a169-e64ee99a0a02 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:14.914431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c4b64-5806-493e-a169-e64ee99a0a02 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:14.914755 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c4b64-5806-493e-a169-e64ee99a0a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:14.943659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b31c4b64-5806-493e-a169-e64ee99a0a02 None None] MFA Rules not processed for user `fa1f21a0d6e342deaa286a8e2bb9ee61`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:14.944930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b31c4b64-5806-493e-a169-e64ee99a0a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:15.016233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1294/2565] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:15.029380 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730c2fa-99ce-4f70-abe1-e7ca714a44c9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:15.031126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c730c2fa-99ce-4f70-abe1-e7ca714a44c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:15.088862 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730c2fa-99ce-4f70-abe1-e7ca714a44c9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:15.092472 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d7905-e6f2-4337-a053-415c67f506d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:15.093137 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d7905-e6f2-4337-a053-415c67f506d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:15.093417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d7905-e6f2-4337-a053-415c67f506d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:15.093692 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d7905-e6f2-4337-a053-415c67f506d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:15.103290 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d7905-e6f2-4337-a053-415c67f506d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:15.104750 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d7905-e6f2-4337-a053-415c67f506d4 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:15.113988 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1272/2566] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:15 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:15.189478 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c4b64-5806-493e-a169-e64ee99a0a02 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:15.199298 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c4b64-5806-493e-a169-e64ee99a0a02 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:15.202931 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380bbccb-9734-4458-9e56-3283cc26448c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:15.203570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bbccb-9734-4458-9e56-3283cc26448c service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:15.203860 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bbccb-9734-4458-9e56-3283cc26448c service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:15.204142 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bbccb-9734-4458-9e56-3283cc26448c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:15.260016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380bbccb-9734-4458-9e56-3283cc26448c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:15.261438 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380bbccb-9734-4458-9e56-3283cc26448c service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:15.270499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1295/2567] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:19.920164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d7905-e6f2-4337-a053-415c67f506d4 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:19.924348 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:19.983080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d7905-e6f2-4337-a053-415c67f506d4 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:19.987697 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804091ad-32cf-4d16-83da-2424fdef7be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:19.988859 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804091ad-32cf-4d16-83da-2424fdef7be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:19.988859 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804091ad-32cf-4d16-83da-2424fdef7be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:19.989108 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804091ad-32cf-4d16-83da-2424fdef7be3 admin admin] PATH_INFO: `/v3/users/9e9543d4ab044991af1e0d5ef4ac1546` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:19.990288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804091ad-32cf-4d16-83da-2424fdef7be3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e9543d4ab044991af1e0d5ef4ac1546)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:19.992192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804091ad-32cf-4d16-83da-2424fdef7be3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:20.039551 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-804091ad-32cf-4d16-83da-2424fdef7be3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e9543d4ab044991af1e0d5ef4ac1546'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:20.040207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-804091ad-32cf-4d16-83da-2424fdef7be3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e9543d4ab044991af1e0d5ef4ac1546'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:20.040840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-804091ad-32cf-4d16-83da-2424fdef7be3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e9543d4ab044991af1e0d5ef4ac1546'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:20.047563 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-804091ad-32cf-4d16-83da-2424fdef7be3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e9543d4ab044991af1e0d5ef4ac1546'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:20.059158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-804091ad-32cf-4d16-83da-2424fdef7be3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e9543d4ab044991af1e0d5ef4ac1546'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:20.071317 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1273/2568] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:36:19 2022] DELETE /identity/v3/users/9e9543d4ab044991af1e0d5ef4ac1546 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:20.076056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1ac3a-e9c6-4fe7-98d5-0930c1290624 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:20.076498 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1ac3a-e9c6-4fe7-98d5-0930c1290624 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:20.076662 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1ac3a-e9c6-4fe7-98d5-0930c1290624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:20.094327 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-24b1ac3a-e9c6-4fe7-98d5-0930c1290624 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:20.095681 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-24b1ac3a-e9c6-4fe7-98d5-0930c1290624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:20.155019 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1296/2569] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:20.164864 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804091ad-32cf-4d16-83da-2424fdef7be3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:20.166257 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-804091ad-32cf-4d16-83da-2424fdef7be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:20.222526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804091ad-32cf-4d16-83da-2424fdef7be3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:20.226159 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da077c-53ab-4480-9ab8-9233065af4ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:20.227060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da077c-53ab-4480-9ab8-9233065af4ac service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:20.227346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da077c-53ab-4480-9ab8-9233065af4ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:20.227624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da077c-53ab-4480-9ab8-9233065af4ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:20.237045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da077c-53ab-4480-9ab8-9233065af4ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:20.238471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da077c-53ab-4480-9ab8-9233065af4ac service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:20.247760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1274/2570] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:20.515468 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1ac3a-e9c6-4fe7-98d5-0930c1290624 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:20.516932 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-24b1ac3a-e9c6-4fe7-98d5-0930c1290624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:20.525228 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:20.529519 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1ac3a-e9c6-4fe7-98d5-0930c1290624 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:20.534612 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fa471-24de-47f0-9ea7-38e501b00b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:20.535563 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fa471-24de-47f0-9ea7-38e501b00b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:20.535922 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fa471-24de-47f0-9ea7-38e501b00b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:20.536276 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fa471-24de-47f0-9ea7-38e501b00b1d admin admin] PATH_INFO: `/v3/projects/e54589f3a8644fdca82daa89ae7d9fa5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:20.537610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032fa471-24de-47f0-9ea7-38e501b00b1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e54589f3a8644fdca82daa89ae7d9fa5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:20.539747 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032fa471-24de-47f0-9ea7-38e501b00b1d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:20.586016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-032fa471-24de-47f0-9ea7-38e501b00b1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e54589f3a8644fdca82daa89ae7d9fa5', 'request_id': 'req-032fa471-24de-47f0-9ea7-38e501b00b1d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:20.591723 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-032fa471-24de-47f0-9ea7-38e501b00b1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e54589f3a8644fdca82daa89ae7d9fa5', 'request_id': 'req-032fa471-24de-47f0-9ea7-38e501b00b1d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:20.598555 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-032fa471-24de-47f0-9ea7-38e501b00b1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e54589f3a8644fdca82daa89ae7d9fa5', 'request_id': 'req-032fa471-24de-47f0-9ea7-38e501b00b1d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:20.612533 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-032fa471-24de-47f0-9ea7-38e501b00b1d admin admin] The token cache is being invalidate because project e54589f3a8644fdca82daa89ae7d9fa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:36:20.612869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-032fa471-24de-47f0-9ea7-38e501b00b1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:20.613498 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-032fa471-24de-47f0-9ea7-38e501b00b1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:20.615470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1297/2571] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:36:20 2022] DELETE /identity/v3/projects/e54589f3a8644fdca82daa89ae7d9fa5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:20.619462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da077c-53ab-4480-9ab8-9233065af4ac service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:20.629375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da077c-53ab-4480-9ab8-9233065af4ac service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:20.650902 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:20.690433 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f83525-8329-4441-852f-935f1ebc1d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:20.691341 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f83525-8329-4441-852f-935f1ebc1d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:20.691627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f83525-8329-4441-852f-935f1ebc1d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:20.691909 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f83525-8329-4441-852f-935f1ebc1d9a admin admin] PATH_INFO: `/v3/users/fa1f21a0d6e342deaa286a8e2bb9ee61` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:20.693157 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f83525-8329-4441-852f-935f1ebc1d9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa1f21a0d6e342deaa286a8e2bb9ee61)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:20.697380 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f83525-8329-4441-852f-935f1ebc1d9a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:20.746314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-29f83525-8329-4441-852f-935f1ebc1d9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa1f21a0d6e342deaa286a8e2bb9ee61'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:20.747208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-29f83525-8329-4441-852f-935f1ebc1d9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa1f21a0d6e342deaa286a8e2bb9ee61'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:20.747872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-29f83525-8329-4441-852f-935f1ebc1d9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa1f21a0d6e342deaa286a8e2bb9ee61'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:20.755120 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-29f83525-8329-4441-852f-935f1ebc1d9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa1f21a0d6e342deaa286a8e2bb9ee61'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:20.766968 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-29f83525-8329-4441-852f-935f1ebc1d9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa1f21a0d6e342deaa286a8e2bb9ee61'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:20.779521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1275/2572] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:36:20 2022] DELETE /identity/v3/users/fa1f21a0d6e342deaa286a8e2bb9ee61 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:21.045799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fa471-24de-47f0-9ea7-38e501b00b1d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:21.047302 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-032fa471-24de-47f0-9ea7-38e501b00b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:21.065045 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fa471-24de-47f0-9ea7-38e501b00b1d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:21.123436 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47792629-d033-4ce7-ad71-4947db943df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:21.124294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47792629-d033-4ce7-ad71-4947db943df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:21.124686 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47792629-d033-4ce7-ad71-4947db943df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:21.124900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47792629-d033-4ce7-ad71-4947db943df1 admin admin] PATH_INFO: `/v3/projects/b38c47d257974538b838e5209ac9da4d` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:21.126109 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47792629-d033-4ce7-ad71-4947db943df1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b38c47d257974538b838e5209ac9da4d)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:21.128008 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47792629-d033-4ce7-ad71-4947db943df1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:21.174610 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-47792629-d033-4ce7-ad71-4947db943df1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b38c47d257974538b838e5209ac9da4d', 'request_id': 'req-47792629-d033-4ce7-ad71-4947db943df1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:21.180064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-47792629-d033-4ce7-ad71-4947db943df1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b38c47d257974538b838e5209ac9da4d', 'request_id': 'req-47792629-d033-4ce7-ad71-4947db943df1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:21.186251 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-47792629-d033-4ce7-ad71-4947db943df1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b38c47d257974538b838e5209ac9da4d', 'request_id': 'req-47792629-d033-4ce7-ad71-4947db943df1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:21.198361 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-47792629-d033-4ce7-ad71-4947db943df1 admin admin] The token cache is being invalidate because project b38c47d257974538b838e5209ac9da4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:36:21.198668 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-47792629-d033-4ce7-ad71-4947db943df1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:21.199266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-47792629-d033-4ce7-ad71-4947db943df1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:21.200830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1298/2573] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:36:21 2022] DELETE /identity/v3/projects/b38c47d257974538b838e5209ac9da4d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:21.206446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ba0dd-9593-4fc1-b90d-1185e1718508 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:21.206948 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ba0dd-9593-4fc1-b90d-1185e1718508 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:21.207051 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ba0dd-9593-4fc1-b90d-1185e1718508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:21.225307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d95ba0dd-9593-4fc1-b90d-1185e1718508 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:21.226750 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d95ba0dd-9593-4fc1-b90d-1185e1718508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:21.283147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1276/2574] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:21.291004 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfa797-b09f-4a80-884e-70b1ce5f0947 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:21.291305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfa797-b09f-4a80-884e-70b1ce5f0947 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:21.291593 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfa797-b09f-4a80-884e-70b1ce5f0947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:21.309809 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9acfa797-b09f-4a80-884e-70b1ce5f0947 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:21.311016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9acfa797-b09f-4a80-884e-70b1ce5f0947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:21.318923 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1299/2575] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:21.323062 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ba0dd-9593-4fc1-b90d-1185e1718508 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:21.333076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ba0dd-9593-4fc1-b90d-1185e1718508 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:21.337533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990db6d6-4012-4a43-b59a-0216cc1c458d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:21.338178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990db6d6-4012-4a43-b59a-0216cc1c458d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:21.338462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990db6d6-4012-4a43-b59a-0216cc1c458d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:21.338737 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990db6d6-4012-4a43-b59a-0216cc1c458d admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:21.339509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990db6d6-4012-4a43-b59a-0216cc1c458d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:36:21.339823 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990db6d6-4012-4a43-b59a-0216cc1c458d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:21.341443 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990db6d6-4012-4a43-b59a-0216cc1c458d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:21.353462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1277/2576] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:36:21.357593 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfa797-b09f-4a80-884e-70b1ce5f0947 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:21.366822 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfa797-b09f-4a80-884e-70b1ce5f0947 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:21.371389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72068470-28da-4672-8705-de42866c7b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:21.372028 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72068470-28da-4672-8705-de42866c7b59 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:21.372312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72068470-28da-4672-8705-de42866c7b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:21.372627 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72068470-28da-4672-8705-de42866c7b59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:21.373317 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72068470-28da-4672-8705-de42866c7b59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:21.375265 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72068470-28da-4672-8705-de42866c7b59 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:21.383696 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:21.403472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1300/2577] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:36:21 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:21.407629 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990db6d6-4012-4a43-b59a-0216cc1c458d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:21.420810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990db6d6-4012-4a43-b59a-0216cc1c458d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:21.452991 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:21.487606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27036741-ed79-4683-ba16-033894642020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:21.488280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27036741-ed79-4683-ba16-033894642020 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:21.488604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27036741-ed79-4683-ba16-033894642020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:21.488890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27036741-ed79-4683-ba16-033894642020 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:21.489595 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27036741-ed79-4683-ba16-033894642020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:21.491537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27036741-ed79-4683-ba16-033894642020 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:21.511033 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1278/2578] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:36:21 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:21.515169 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72068470-28da-4672-8705-de42866c7b59 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:21.525972 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72068470-28da-4672-8705-de42866c7b59 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:21.530566 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a658e8c-2974-4338-8c17-112a8c377fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:21.531218 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a658e8c-2974-4338-8c17-112a8c377fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:21.531507 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a658e8c-2974-4338-8c17-112a8c377fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:21.531787 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a658e8c-2974-4338-8c17-112a8c377fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:21.532530 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a658e8c-2974-4338-8c17-112a8c377fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:21.534151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a658e8c-2974-4338-8c17-112a8c377fd7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:21.544142 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1301/2579] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:21.548160 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27036741-ed79-4683-ba16-033894642020 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:21.558029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27036741-ed79-4683-ba16-033894642020 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:21.562743 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d6f14-f251-48dc-bea3-79f44c707245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:21.563412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d6f14-f251-48dc-bea3-79f44c707245 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:21.564216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d6f14-f251-48dc-bea3-79f44c707245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:21.564518 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d6f14-f251-48dc-bea3-79f44c707245 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:21.565252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d6f14-f251-48dc-bea3-79f44c707245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:21.566852 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d6f14-f251-48dc-bea3-79f44c707245 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:21.576792 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1279/2580] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:21.581438 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a658e8c-2974-4338-8c17-112a8c377fd7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:21.591219 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a658e8c-2974-4338-8c17-112a8c377fd7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:21.596182 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4b314-a582-4b5f-8d57-46ec56b97b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:21.597116 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4b314-a582-4b5f-8d57-46ec56b97b1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:21.597400 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4b314-a582-4b5f-8d57-46ec56b97b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:21.597676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4b314-a582-4b5f-8d57-46ec56b97b1d admin admin] PATH_INFO: `/v3/projects/f4b2130f376047ca9abaf13c80ea2453/users/b2390f897aeb4f8081c4a72cb3f1b285/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:21.612147 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4b314-a582-4b5f-8d57-46ec56b97b1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b2130f376047ca9abaf13c80ea2453, user_id=b2390f897aeb4f8081c4a72cb3f1b285, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:21.614839 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4b314-a582-4b5f-8d57-46ec56b97b1d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:21.631502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1302/2581] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:21 2022] PUT /identity/v3/projects/f4b2130f376047ca9abaf13c80ea2453/users/b2390f897aeb4f8081c4a72cb3f1b285/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:21.643620 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a9c02-1f27-4ea2-9107-6f15ce721887 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:21.643919 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a9c02-1f27-4ea2-9107-6f15ce721887 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:21.644211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a9c02-1f27-4ea2-9107-6f15ce721887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:21.670109 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a16a9c02-1f27-4ea2-9107-6f15ce721887 None None] MFA Rules not processed for user `b2390f897aeb4f8081c4a72cb3f1b285`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:21.671320 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a16a9c02-1f27-4ea2-9107-6f15ce721887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:21.733736 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1280/2582] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:36:21.745484 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4b314-a582-4b5f-8d57-46ec56b97b1d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:21.747252 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-eea4b314-a582-4b5f-8d57-46ec56b97b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:21.807218 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4b314-a582-4b5f-8d57-46ec56b97b1d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:21.812927 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dbc3ab-bd94-47a6-a1f4-1ee0756ecd35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:21.813672 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbc3ab-bd94-47a6-a1f4-1ee0756ecd35 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:21.813955 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbc3ab-bd94-47a6-a1f4-1ee0756ecd35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:21.814250 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbc3ab-bd94-47a6-a1f4-1ee0756ecd35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:21.825435 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbc3ab-bd94-47a6-a1f4-1ee0756ecd35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:21.826934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dbc3ab-bd94-47a6-a1f4-1ee0756ecd35 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:21.836170 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1303/2583] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:31.730825 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=52,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:31.832528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=45,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:46.608526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a9c02-1f27-4ea2-9107-6f15ce721887 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:46.672381 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a9c02-1f27-4ea2-9107-6f15ce721887 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:46.677383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1168aa-b0a0-4698-b0c4-a72d52752552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:46.678275 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1168aa-b0a0-4698-b0c4-a72d52752552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:46.678563 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1168aa-b0a0-4698-b0c4-a72d52752552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:46.678831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1168aa-b0a0-4698-b0c4-a72d52752552 admin admin] PATH_INFO: `/v3/users/b2390f897aeb4f8081c4a72cb3f1b285` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:46.680040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1168aa-b0a0-4698-b0c4-a72d52752552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2390f897aeb4f8081c4a72cb3f1b285)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:46.682003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1168aa-b0a0-4698-b0c4-a72d52752552 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:46.728296 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c1168aa-b0a0-4698-b0c4-a72d52752552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2390f897aeb4f8081c4a72cb3f1b285'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:46.728928 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c1168aa-b0a0-4698-b0c4-a72d52752552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2390f897aeb4f8081c4a72cb3f1b285'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:46.729513 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c1168aa-b0a0-4698-b0c4-a72d52752552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2390f897aeb4f8081c4a72cb3f1b285'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:46.737339 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c1168aa-b0a0-4698-b0c4-a72d52752552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2390f897aeb4f8081c4a72cb3f1b285'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:46.750433 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5c1168aa-b0a0-4698-b0c4-a72d52752552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2390f897aeb4f8081c4a72cb3f1b285'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:46.769309 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1281/2584] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:36:46 2022] DELETE /identity/v3/users/b2390f897aeb4f8081c4a72cb3f1b285 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:46.774240 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43710385-20ab-48d1-8417-93330b2551ba None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:46.774742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43710385-20ab-48d1-8417-93330b2551ba None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:46.774850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43710385-20ab-48d1-8417-93330b2551ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:46.795286 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-43710385-20ab-48d1-8417-93330b2551ba None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:46.796489 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-43710385-20ab-48d1-8417-93330b2551ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:46.859779 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1304/2585] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:46.869896 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1168aa-b0a0-4698-b0c4-a72d52752552 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:46.871499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c1168aa-b0a0-4698-b0c4-a72d52752552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:46.933504 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1168aa-b0a0-4698-b0c4-a72d52752552 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:46.937425 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9736e-8beb-44ea-a0c7-057f7ca127e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:46.938071 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9736e-8beb-44ea-a0c7-057f7ca127e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:46.938385 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9736e-8beb-44ea-a0c7-057f7ca127e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:46.938659 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9736e-8beb-44ea-a0c7-057f7ca127e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:46.947623 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9736e-8beb-44ea-a0c7-057f7ca127e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:46.949094 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9736e-8beb-44ea-a0c7-057f7ca127e9 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:46.958413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1282/2586] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:47.246095 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43710385-20ab-48d1-8417-93330b2551ba None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:47.247703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-43710385-20ab-48d1-8417-93330b2551ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:47.259874 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43710385-20ab-48d1-8417-93330b2551ba None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:47.265981 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6af15-e2c4-4ba0-870d-833fe74a6091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:47.266856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6af15-e2c4-4ba0-870d-833fe74a6091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:47.267143 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6af15-e2c4-4ba0-870d-833fe74a6091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:47.267430 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6af15-e2c4-4ba0-870d-833fe74a6091 admin admin] PATH_INFO: `/v3/projects/f4b2130f376047ca9abaf13c80ea2453` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:47.268687 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6af15-e2c4-4ba0-870d-833fe74a6091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b2130f376047ca9abaf13c80ea2453)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:47.270638 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6af15-e2c4-4ba0-870d-833fe74a6091 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:47.321560 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8ef6af15-e2c4-4ba0-870d-833fe74a6091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b2130f376047ca9abaf13c80ea2453', 'request_id': 'req-8ef6af15-e2c4-4ba0-870d-833fe74a6091'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:47.327540 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8ef6af15-e2c4-4ba0-870d-833fe74a6091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b2130f376047ca9abaf13c80ea2453', 'request_id': 'req-8ef6af15-e2c4-4ba0-870d-833fe74a6091'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:47.334269 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8ef6af15-e2c4-4ba0-870d-833fe74a6091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b2130f376047ca9abaf13c80ea2453', 'request_id': 'req-8ef6af15-e2c4-4ba0-870d-833fe74a6091'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:47.346654 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8ef6af15-e2c4-4ba0-870d-833fe74a6091 admin admin] The token cache is being invalidate because project f4b2130f376047ca9abaf13c80ea2453 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:36:47.346946 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8ef6af15-e2c4-4ba0-870d-833fe74a6091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:47.347599 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8ef6af15-e2c4-4ba0-870d-833fe74a6091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:47.349830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1305/2587] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:36:47 2022] DELETE /identity/v3/projects/f4b2130f376047ca9abaf13c80ea2453 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:47.354956 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d90b4d-648a-452b-b664-5534fbc9869f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:47.355252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d90b4d-648a-452b-b664-5534fbc9869f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:47.355534 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d90b4d-648a-452b-b664-5534fbc9869f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:47.373538 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b9d90b4d-648a-452b-b664-5534fbc9869f None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:47.374738 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b9d90b4d-648a-452b-b664-5534fbc9869f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:47.391082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:47.434804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1283/2588] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:47.442605 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e0d4e-cf78-4a16-9916-292fc5b15fe9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:47.442925 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e0d4e-cf78-4a16-9916-292fc5b15fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:47.443262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e0d4e-cf78-4a16-9916-292fc5b15fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:47.461748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c41e0d4e-cf78-4a16-9916-292fc5b15fe9 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:47.462954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c41e0d4e-cf78-4a16-9916-292fc5b15fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:47.470959 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1306/2589] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:47.475647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d90b4d-648a-452b-b664-5534fbc9869f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:47.485856 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d90b4d-648a-452b-b664-5534fbc9869f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:47.490528 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fafacf-345e-45b6-94aa-f5f0548b5504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:47.491207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fafacf-345e-45b6-94aa-f5f0548b5504 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:47.492105 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fafacf-345e-45b6-94aa-f5f0548b5504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:47.492442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fafacf-345e-45b6-94aa-f5f0548b5504 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:47.493157 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fafacf-345e-45b6-94aa-f5f0548b5504 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:36:47.493476 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fafacf-345e-45b6-94aa-f5f0548b5504 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:47.495078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fafacf-345e-45b6-94aa-f5f0548b5504 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:47.507073 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1284/2590] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:36:47.511422 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e0d4e-cf78-4a16-9916-292fc5b15fe9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:47.520538 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e0d4e-cf78-4a16-9916-292fc5b15fe9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:47.527613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c66ac5-be91-4b89-90d0-63e6ecac8bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:47.528270 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c66ac5-be91-4b89-90d0-63e6ecac8bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:47.528580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c66ac5-be91-4b89-90d0-63e6ecac8bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:47.528858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c66ac5-be91-4b89-90d0-63e6ecac8bf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:47.529558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c66ac5-be91-4b89-90d0-63e6ecac8bf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:47.531461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c66ac5-be91-4b89-90d0-63e6ecac8bf2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:47.558004 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1307/2591] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:36:47 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:47.562314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fafacf-345e-45b6-94aa-f5f0548b5504 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:47.572515 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fafacf-345e-45b6-94aa-f5f0548b5504 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:47.630845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841d7bb-a30f-4db4-9fe5-06b254e3ea46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:47.631588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841d7bb-a30f-4db4-9fe5-06b254e3ea46 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:47.631867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841d7bb-a30f-4db4-9fe5-06b254e3ea46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:47.632247 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841d7bb-a30f-4db4-9fe5-06b254e3ea46 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:47.632788 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841d7bb-a30f-4db4-9fe5-06b254e3ea46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:47.634777 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841d7bb-a30f-4db4-9fe5-06b254e3ea46 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:47.662984 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1285/2592] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:36:47 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:47.667455 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c66ac5-be91-4b89-90d0-63e6ecac8bf2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:47.678570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c66ac5-be91-4b89-90d0-63e6ecac8bf2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:47.683207 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23774c-dc35-4fd4-b887-fff4a5f2ceb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:47.683865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23774c-dc35-4fd4-b887-fff4a5f2ceb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:47.684151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23774c-dc35-4fd4-b887-fff4a5f2ceb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:47.684431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23774c-dc35-4fd4-b887-fff4a5f2ceb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:47.685190 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23774c-dc35-4fd4-b887-fff4a5f2ceb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:47.686785 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23774c-dc35-4fd4-b887-fff4a5f2ceb1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:47.697016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1308/2593] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:47.700948 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841d7bb-a30f-4db4-9fe5-06b254e3ea46 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:47.710904 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841d7bb-a30f-4db4-9fe5-06b254e3ea46 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:47.715498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed1d66-cb2c-438b-a9af-3caf548fb849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:47.716176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed1d66-cb2c-438b-a9af-3caf548fb849 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:47.716462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed1d66-cb2c-438b-a9af-3caf548fb849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:47.716772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed1d66-cb2c-438b-a9af-3caf548fb849 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:47.717501 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed1d66-cb2c-438b-a9af-3caf548fb849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:47.719087 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed1d66-cb2c-438b-a9af-3caf548fb849 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:47.729816 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1286/2594] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:47.733714 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23774c-dc35-4fd4-b887-fff4a5f2ceb1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:47.744083 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23774c-dc35-4fd4-b887-fff4a5f2ceb1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:47.748563 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572a07b-25c1-4afc-8a75-242ce668b751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:47.749432 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572a07b-25c1-4afc-8a75-242ce668b751 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:47.749716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572a07b-25c1-4afc-8a75-242ce668b751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:47.749990 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572a07b-25c1-4afc-8a75-242ce668b751 admin admin] PATH_INFO: `/v3/projects/86e658ce6d174f678a9e7a67da6129ad/users/bcc941012f144612959498eafbae030c/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:47.763960 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572a07b-25c1-4afc-8a75-242ce668b751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86e658ce6d174f678a9e7a67da6129ad, user_id=bcc941012f144612959498eafbae030c, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:47.766642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572a07b-25c1-4afc-8a75-242ce668b751 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:47.779140 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1309/2595] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:47 2022] PUT /identity/v3/projects/86e658ce6d174f678a9e7a67da6129ad/users/bcc941012f144612959498eafbae030c/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:47.790891 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bf22e-1409-48f9-9c28-974d333268be None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:47.791194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bf22e-1409-48f9-9c28-974d333268be None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:47.791481 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bf22e-1409-48f9-9c28-974d333268be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:47.821220 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-874bf22e-1409-48f9-9c28-974d333268be None None] MFA Rules not processed for user `bcc941012f144612959498eafbae030c`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:47.822445 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-874bf22e-1409-48f9-9c28-974d333268be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:47.886759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1287/2596] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:47.891512 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572a07b-25c1-4afc-8a75-242ce668b751 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:47.914420 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:47.957846 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572a07b-25c1-4afc-8a75-242ce668b751 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:47.963048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3152286-bd4f-4260-9b28-46055e215960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:47.963704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3152286-bd4f-4260-9b28-46055e215960 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:47.963999 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3152286-bd4f-4260-9b28-46055e215960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:47.964279 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3152286-bd4f-4260-9b28-46055e215960 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:47.965002 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3152286-bd4f-4260-9b28-46055e215960 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:47.966908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3152286-bd4f-4260-9b28-46055e215960 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:47.992674 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1310/2597] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:36:47 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:36:47.996711 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bf22e-1409-48f9-9c28-974d333268be None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:48.007206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bf22e-1409-48f9-9c28-974d333268be None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:48.019126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:48.063267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbc26a-e2aa-4fcc-9e23-28e4d745c99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:48.063933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbc26a-e2aa-4fcc-9e23-28e4d745c99f admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.064223 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbc26a-e2aa-4fcc-9e23-28e4d745c99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.064518 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbc26a-e2aa-4fcc-9e23-28e4d745c99f admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.065221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbc26a-e2aa-4fcc-9e23-28e4d745c99f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:48.067159 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbc26a-e2aa-4fcc-9e23-28e4d745c99f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:48.093281 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1288/2598] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:36:47 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:48.094484 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3152286-bd4f-4260-9b28-46055e215960 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:48.105750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3152286-bd4f-4260-9b28-46055e215960 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:48.110521 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44cead-5e09-421c-8690-e9b068de932b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:48.111188 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44cead-5e09-421c-8690-e9b068de932b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.111475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44cead-5e09-421c-8690-e9b068de932b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.111754 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44cead-5e09-421c-8690-e9b068de932b admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.112486 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44cead-5e09-421c-8690-e9b068de932b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:48.114121 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44cead-5e09-421c-8690-e9b068de932b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:48.125681 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1311/2599] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:48.129722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbc26a-e2aa-4fcc-9e23-28e4d745c99f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:48.139907 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbc26a-e2aa-4fcc-9e23-28e4d745c99f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:48.145584 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6db28-915f-40c5-8444-4173659fd95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:48.146508 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6db28-915f-40c5-8444-4173659fd95d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.146799 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6db28-915f-40c5-8444-4173659fd95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.147083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6db28-915f-40c5-8444-4173659fd95d admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.147824 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6db28-915f-40c5-8444-4173659fd95d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:48.149467 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6db28-915f-40c5-8444-4173659fd95d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:48.160980 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1289/2600] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:36:48.167483 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44cead-5e09-421c-8690-e9b068de932b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:48.176086 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44cead-5e09-421c-8690-e9b068de932b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:48.181629 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f97e7-3f63-4106-9c80-865940cfdc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:48.182548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f97e7-3f63-4106-9c80-865940cfdc22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.182840 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f97e7-3f63-4106-9c80-865940cfdc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.183510 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f97e7-3f63-4106-9c80-865940cfdc22 admin admin] PATH_INFO: `/v3/projects/dcb37bf146a348c2acd4a998f24f6ddb/users/4c480c73b3d748959c7f206325de2b8e/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.198260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f97e7-3f63-4106-9c80-865940cfdc22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcb37bf146a348c2acd4a998f24f6ddb, user_id=4c480c73b3d748959c7f206325de2b8e, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:48.201010 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f97e7-3f63-4106-9c80-865940cfdc22 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:48.217290 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1312/2601] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:48 2022] PUT /identity/v3/projects/dcb37bf146a348c2acd4a998f24f6ddb/users/4c480c73b3d748959c7f206325de2b8e/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:48.228331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84148b9f-0e19-4c94-881d-36d9dadb0a10 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.228644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84148b9f-0e19-4c94-881d-36d9dadb0a10 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.228922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84148b9f-0e19-4c94-881d-36d9dadb0a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.256526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-84148b9f-0e19-4c94-881d-36d9dadb0a10 None None] MFA Rules not processed for user `4c480c73b3d748959c7f206325de2b8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:48.257705 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-84148b9f-0e19-4c94-881d-36d9dadb0a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:48.323715 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1290/2602] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:48.328311 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f97e7-3f63-4106-9c80-865940cfdc22 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:48.394583 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f97e7-3f63-4106-9c80-865940cfdc22 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:48.400117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de3315-a261-4396-8d94-4491494556d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:48.400872 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de3315-a261-4396-8d94-4491494556d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.401165 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de3315-a261-4396-8d94-4491494556d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.401450 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de3315-a261-4396-8d94-4491494556d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.402161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de3315-a261-4396-8d94-4491494556d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:48.404080 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de3315-a261-4396-8d94-4491494556d1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:48.432314 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1313/2603] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:36:48 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:48.436234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84148b9f-0e19-4c94-881d-36d9dadb0a10 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:48.445221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de3315-a261-4396-8d94-4491494556d1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:48.446896 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e3de3315-a261-4396-8d94-4491494556d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:48.448489 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84148b9f-0e19-4c94-881d-36d9dadb0a10 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:48.507451 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b93242-64e4-42a6-911a-b0461ca917b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:48.507681 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:48.508207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b93242-64e4-42a6-911a-b0461ca917b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.508490 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b93242-64e4-42a6-911a-b0461ca917b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.508812 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b93242-64e4-42a6-911a-b0461ca917b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.509514 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b93242-64e4-42a6-911a-b0461ca917b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:48.511526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b93242-64e4-42a6-911a-b0461ca917b6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:48.513146 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de3315-a261-4396-8d94-4491494556d1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:48.518423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27a2c7-e193-428f-9866-6b90616930b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:48.519334 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27a2c7-e193-428f-9866-6b90616930b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.519624 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27a2c7-e193-428f-9866-6b90616930b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.519910 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27a2c7-e193-428f-9866-6b90616930b3 admin admin] PATH_INFO: `/v3/users/6960f1d6ff0747d3addabd96dc6df651` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.521781 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27a2c7-e193-428f-9866-6b90616930b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6960f1d6ff0747d3addabd96dc6df651)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:48.523710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27a2c7-e193-428f-9866-6b90616930b3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:48.531631 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1291/2604] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:36:48 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:48.535657 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b93242-64e4-42a6-911a-b0461ca917b6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:48.547031 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b93242-64e4-42a6-911a-b0461ca917b6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:48.551836 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e78a02-c916-451e-b030-fbfdcfd7e71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:48.553082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e78a02-c916-451e-b030-fbfdcfd7e71a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.553399 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e78a02-c916-451e-b030-fbfdcfd7e71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.553641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e78a02-c916-451e-b030-fbfdcfd7e71a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.554368 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e78a02-c916-451e-b030-fbfdcfd7e71a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:48.555967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e78a02-c916-451e-b030-fbfdcfd7e71a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:48.569806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1292/2605] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:36:48.570834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e78a02-c916-451e-b030-fbfdcfd7e71a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:48.577279 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:48.579690 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fe27a2c7-e193-428f-9866-6b90616930b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6960f1d6ff0747d3addabd96dc6df651'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:48.581778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e78a02-c916-451e-b030-fbfdcfd7e71a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:48.588210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b872b-f3e0-4277-849c-aa75bbd67489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:48.589251 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b872b-f3e0-4277-849c-aa75bbd67489 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.589543 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b872b-f3e0-4277-849c-aa75bbd67489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.589821 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b872b-f3e0-4277-849c-aa75bbd67489 admin admin] PATH_INFO: `/v3/projects/f3d77150222e41778be09a33b20a905b/users/1990f1d45aa841e69f0c6d65011a9be3/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.592800 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fe27a2c7-e193-428f-9866-6b90616930b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6960f1d6ff0747d3addabd96dc6df651'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:48.593538 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fe27a2c7-e193-428f-9866-6b90616930b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6960f1d6ff0747d3addabd96dc6df651'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:48.594172 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fe27a2c7-e193-428f-9866-6b90616930b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6960f1d6ff0747d3addabd96dc6df651'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:48.602030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fe27a2c7-e193-428f-9866-6b90616930b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6960f1d6ff0747d3addabd96dc6df651'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:48.606706 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b872b-f3e0-4277-849c-aa75bbd67489 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3d77150222e41778be09a33b20a905b, user_id=1990f1d45aa841e69f0c6d65011a9be3, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:48.609445 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b872b-f3e0-4277-849c-aa75bbd67489 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:48.616394 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1314/2606] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:36:48 2022] DELETE /identity/v3/users/6960f1d6ff0747d3addabd96dc6df651 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:48.621062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb497ae4-344d-42b4-a7ab-3ba02d84fe95 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.621563 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb497ae4-344d-42b4-a7ab-3ba02d84fe95 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.621847 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb497ae4-344d-42b4-a7ab-3ba02d84fe95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.623171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1293/2607] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:48 2022] PUT /identity/v3/projects/f3d77150222e41778be09a33b20a905b/users/1990f1d45aa841e69f0c6d65011a9be3/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:48.626884 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b872b-f3e0-4277-849c-aa75bbd67489 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:48.628499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-537b872b-f3e0-4277-849c-aa75bbd67489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:48.640029 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bb497ae4-344d-42b4-a7ab-3ba02d84fe95 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:48.641256 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bb497ae4-344d-42b4-a7ab-3ba02d84fe95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:48.691488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b872b-f3e0-4277-849c-aa75bbd67489 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:48.696793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d166be-4249-4075-ae04-a0130ebf0c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:48.697449 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d166be-4249-4075-ae04-a0130ebf0c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.697730 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d166be-4249-4075-ae04-a0130ebf0c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.698004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d166be-4249-4075-ae04-a0130ebf0c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.698788 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d166be-4249-4075-ae04-a0130ebf0c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:48.700308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d166be-4249-4075-ae04-a0130ebf0c2a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:48.701150 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1315/2608] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:48.710433 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1294/2609] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:48.711469 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb497ae4-344d-42b4-a7ab-3ba02d84fe95 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:48.713056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bb497ae4-344d-42b4-a7ab-3ba02d84fe95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:48.714186 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d166be-4249-4075-ae04-a0130ebf0c2a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:48.724242 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d166be-4249-4075-ae04-a0130ebf0c2a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:48.729057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c48ce-f66e-4d90-b1f8-895a104fc72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:48.729916 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c48ce-f66e-4d90-b1f8-895a104fc72b admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.730200 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c48ce-f66e-4d90-b1f8-895a104fc72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.730472 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c48ce-f66e-4d90-b1f8-895a104fc72b admin admin] PATH_INFO: `/v3/projects/f3d77150222e41778be09a33b20a905b/users/1990f1d45aa841e69f0c6d65011a9be3/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.732441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c48ce-f66e-4d90-b1f8-895a104fc72b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3d77150222e41778be09a33b20a905b, user_id=1990f1d45aa841e69f0c6d65011a9be3, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:48.735108 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c48ce-f66e-4d90-b1f8-895a104fc72b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:48.745666 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1295/2610] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:48 2022] PUT /identity/v3/projects/f3d77150222e41778be09a33b20a905b/users/1990f1d45aa841e69f0c6d65011a9be3/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:48.756227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df6a9c-93a0-46bc-98da-edb915496356 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.756563 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df6a9c-93a0-46bc-98da-edb915496356 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.756821 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df6a9c-93a0-46bc-98da-edb915496356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.773968 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb497ae4-344d-42b4-a7ab-3ba02d84fe95 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:48.782426 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b7df6a9c-93a0-46bc-98da-edb915496356 None None] MFA Rules not processed for user `1990f1d45aa841e69f0c6d65011a9be3`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:48.783704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b7df6a9c-93a0-46bc-98da-edb915496356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:48.804082 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:48.827347 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb41cee-4e7e-4acf-9d61-e93706b24a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:48.828001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb41cee-4e7e-4acf-9d61-e93706b24a86 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.828291 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb41cee-4e7e-4acf-9d61-e93706b24a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.828608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb41cee-4e7e-4acf-9d61-e93706b24a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.854006 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1296/2611] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:36:48.865443 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df6a9c-93a0-46bc-98da-edb915496356 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:48.866842 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b7df6a9c-93a0-46bc-98da-edb915496356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:48.890266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb41cee-4e7e-4acf-9d61-e93706b24a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:48.891708 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb41cee-4e7e-4acf-9d61-e93706b24a86 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:48.901452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1316/2612] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:48.918456 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:48.922929 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df6a9c-93a0-46bc-98da-edb915496356 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:48.928334 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd8730-6662-4281-9436-b2deee3e621e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:48.929012 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd8730-6662-4281-9436-b2deee3e621e service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:48.929295 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd8730-6662-4281-9436-b2deee3e621e service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:48.929571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd8730-6662-4281-9436-b2deee3e621e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:48.931359 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4abd8730-6662-4281-9436-b2deee3e621e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:48.992702 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd8730-6662-4281-9436-b2deee3e621e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:48.994223 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd8730-6662-4281-9436-b2deee3e621e service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:49.004116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1297/2613] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:36:49.180065 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb41cee-4e7e-4acf-9d61-e93706b24a86 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:49.180065 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4bb41cee-4e7e-4acf-9d61-e93706b24a86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:49.193485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb41cee-4e7e-4acf-9d61-e93706b24a86 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:49.198658 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da7c57-c179-44bf-994c-9f07e037fdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:49.199617 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da7c57-c179-44bf-994c-9f07e037fdf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:49.199936 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da7c57-c179-44bf-994c-9f07e037fdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:49.200215 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da7c57-c179-44bf-994c-9f07e037fdf1 admin admin] PATH_INFO: `/v3/projects/06860d35e56541e89e34db060cae0347` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:49.201467 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da7c57-c179-44bf-994c-9f07e037fdf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06860d35e56541e89e34db060cae0347)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:49.203771 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da7c57-c179-44bf-994c-9f07e037fdf1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:49.254207 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f9da7c57-c179-44bf-994c-9f07e037fdf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06860d35e56541e89e34db060cae0347', 'request_id': 'req-f9da7c57-c179-44bf-994c-9f07e037fdf1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:49.261001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f9da7c57-c179-44bf-994c-9f07e037fdf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06860d35e56541e89e34db060cae0347', 'request_id': 'req-f9da7c57-c179-44bf-994c-9f07e037fdf1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:49.267605 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f9da7c57-c179-44bf-994c-9f07e037fdf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06860d35e56541e89e34db060cae0347', 'request_id': 'req-f9da7c57-c179-44bf-994c-9f07e037fdf1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:49.280550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f9da7c57-c179-44bf-994c-9f07e037fdf1 admin admin] The token cache is being invalidate because project 06860d35e56541e89e34db060cae0347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:36:49.280947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f9da7c57-c179-44bf-994c-9f07e037fdf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:49.281673 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f9da7c57-c179-44bf-994c-9f07e037fdf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:49.287091 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1317/2614] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:36:49 2022] DELETE /identity/v3/projects/06860d35e56541e89e34db060cae0347 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:49.288847 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd8730-6662-4281-9436-b2deee3e621e service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:49.301358 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd8730-6662-4281-9436-b2deee3e621e service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:49.372291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765b94c-6760-47c5-b3ae-fafea41d12e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:49.373207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765b94c-6760-47c5-b3ae-fafea41d12e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:49.373495 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765b94c-6760-47c5-b3ae-fafea41d12e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:49.373777 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765b94c-6760-47c5-b3ae-fafea41d12e9 admin admin] PATH_INFO: `/v3/users/e544c53462d745bfa62325cfcbe92a8a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:49.374982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765b94c-6760-47c5-b3ae-fafea41d12e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e544c53462d745bfa62325cfcbe92a8a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:49.376962 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765b94c-6760-47c5-b3ae-fafea41d12e9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:49.424057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2765b94c-6760-47c5-b3ae-fafea41d12e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e544c53462d745bfa62325cfcbe92a8a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:49.424741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2765b94c-6760-47c5-b3ae-fafea41d12e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e544c53462d745bfa62325cfcbe92a8a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:49.425352 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2765b94c-6760-47c5-b3ae-fafea41d12e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e544c53462d745bfa62325cfcbe92a8a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:49.432144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2765b94c-6760-47c5-b3ae-fafea41d12e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e544c53462d745bfa62325cfcbe92a8a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:49.444131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2765b94c-6760-47c5-b3ae-fafea41d12e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e544c53462d745bfa62325cfcbe92a8a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:49.456090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1298/2615] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:36:49 2022] DELETE /identity/v3/users/e544c53462d745bfa62325cfcbe92a8a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:49.736587 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da7c57-c179-44bf-994c-9f07e037fdf1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:49.737978 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f9da7c57-c179-44bf-994c-9f07e037fdf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:49.750769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da7c57-c179-44bf-994c-9f07e037fdf1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:49.807617 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e69d2e-a6ce-41c8-81fa-912d0d71cb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:49.808401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e69d2e-a6ce-41c8-81fa-912d0d71cb45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:49.808708 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e69d2e-a6ce-41c8-81fa-912d0d71cb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:49.808983 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e69d2e-a6ce-41c8-81fa-912d0d71cb45 admin admin] PATH_INFO: `/v3/projects/93898a2c71534bd6ab92c09516512659` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:49.810142 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e69d2e-a6ce-41c8-81fa-912d0d71cb45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93898a2c71534bd6ab92c09516512659)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:49.812048 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e69d2e-a6ce-41c8-81fa-912d0d71cb45 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:49.818693 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:49.860280 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-52e69d2e-a6ce-41c8-81fa-912d0d71cb45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93898a2c71534bd6ab92c09516512659', 'request_id': 'req-52e69d2e-a6ce-41c8-81fa-912d0d71cb45'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:49.865756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-52e69d2e-a6ce-41c8-81fa-912d0d71cb45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93898a2c71534bd6ab92c09516512659', 'request_id': 'req-52e69d2e-a6ce-41c8-81fa-912d0d71cb45'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:49.872253 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-52e69d2e-a6ce-41c8-81fa-912d0d71cb45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93898a2c71534bd6ab92c09516512659', 'request_id': 'req-52e69d2e-a6ce-41c8-81fa-912d0d71cb45'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:49.898515 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-52e69d2e-a6ce-41c8-81fa-912d0d71cb45 admin admin] The token cache is being invalidate because project 93898a2c71534bd6ab92c09516512659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:36:49.898805 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-52e69d2e-a6ce-41c8-81fa-912d0d71cb45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:49.899447 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-52e69d2e-a6ce-41c8-81fa-912d0d71cb45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:49.901027 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1318/2616] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:36:49 2022] DELETE /identity/v3/projects/93898a2c71534bd6ab92c09516512659 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:49.905986 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8fa34-9ff9-4f5b-a034-53a1df2a2bc9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:49.906532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8fa34-9ff9-4f5b-a034-53a1df2a2bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:49.906664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8fa34-9ff9-4f5b-a034-53a1df2a2bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:49.931561 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:49.972627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bdd8fa34-9ff9-4f5b-a034-53a1df2a2bc9 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:49.973828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bdd8fa34-9ff9-4f5b-a034-53a1df2a2bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:50.079237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1299/2617] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:50.088160 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a184112-a4f8-4ec9-9992-7e03d9cf8bd9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:50.088453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a184112-a4f8-4ec9-9992-7e03d9cf8bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:50.088764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a184112-a4f8-4ec9-9992-7e03d9cf8bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:50.106575 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9a184112-a4f8-4ec9-9992-7e03d9cf8bd9 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:50.107773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9a184112-a4f8-4ec9-9992-7e03d9cf8bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:50.116025 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1319/2618] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:36:50.120069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8fa34-9ff9-4f5b-a034-53a1df2a2bc9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:50.129529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8fa34-9ff9-4f5b-a034-53a1df2a2bc9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:50.134629 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1b75c-4156-409e-af1f-b57a536cfee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:50.135277 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1b75c-4156-409e-af1f-b57a536cfee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:50.135561 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1b75c-4156-409e-af1f-b57a536cfee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:50.135839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1b75c-4156-409e-af1f-b57a536cfee2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:50.136624 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1b75c-4156-409e-af1f-b57a536cfee2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:36:50.136930 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1b75c-4156-409e-af1f-b57a536cfee2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:50.138515 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1b75c-4156-409e-af1f-b57a536cfee2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:50.150062 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1300/2619] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:36:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:36:50.154530 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a184112-a4f8-4ec9-9992-7e03d9cf8bd9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:50.164060 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a184112-a4f8-4ec9-9992-7e03d9cf8bd9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:50.168703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbefe68-c2c8-4228-a1bf-70a467ae2deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:50.169378 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbefe68-c2c8-4228-a1bf-70a467ae2deb admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:50.169667 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbefe68-c2c8-4228-a1bf-70a467ae2deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:50.169952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbefe68-c2c8-4228-a1bf-70a467ae2deb admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:50.170655 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbefe68-c2c8-4228-a1bf-70a467ae2deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:50.172575 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbefe68-c2c8-4228-a1bf-70a467ae2deb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:50.199599 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1320/2620] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:36:50 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:50.203637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1b75c-4156-409e-af1f-b57a536cfee2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:50.213571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1b75c-4156-409e-af1f-b57a536cfee2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:50.268996 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5b12a-616b-41e5-8b52-cc125cb6590a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:50.269659 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5b12a-616b-41e5-8b52-cc125cb6590a admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:50.269944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5b12a-616b-41e5-8b52-cc125cb6590a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:50.270223 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5b12a-616b-41e5-8b52-cc125cb6590a admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:50.270909 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef5b12a-616b-41e5-8b52-cc125cb6590a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:50.272849 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef5b12a-616b-41e5-8b52-cc125cb6590a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:50.292220 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1301/2621] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:36:50 2022] POST /identity/v3/users => generated 282 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:50.296344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbefe68-c2c8-4228-a1bf-70a467ae2deb admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:50.307120 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbefe68-c2c8-4228-a1bf-70a467ae2deb admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:50.312186 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae29e4-7a7c-42dc-b6ed-633b4d6330e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:50.312875 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae29e4-7a7c-42dc-b6ed-633b4d6330e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:50.313163 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae29e4-7a7c-42dc-b6ed-633b4d6330e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:50.313439 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae29e4-7a7c-42dc-b6ed-633b4d6330e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:50.314168 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae29e4-7a7c-42dc-b6ed-633b4d6330e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:50.315753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae29e4-7a7c-42dc-b6ed-633b4d6330e5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:50.329651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1321/2622] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:50.330860 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5b12a-616b-41e5-8b52-cc125cb6590a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:50.341096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5b12a-616b-41e5-8b52-cc125cb6590a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:50.345973 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7195970-3d79-4550-b236-2f63c9393ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:50.346662 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7195970-3d79-4550-b236-2f63c9393ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:50.346955 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7195970-3d79-4550-b236-2f63c9393ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:50.347239 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7195970-3d79-4550-b236-2f63c9393ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:50.347972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7195970-3d79-4550-b236-2f63c9393ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:50.349696 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7195970-3d79-4550-b236-2f63c9393ab4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:50.360383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1302/2623] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:50.369778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae29e4-7a7c-42dc-b6ed-633b4d6330e5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:50.379783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae29e4-7a7c-42dc-b6ed-633b4d6330e5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:50.385316 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac027aa-936c-41f9-a6a8-d3850f01fd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:50.386206 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac027aa-936c-41f9-a6a8-d3850f01fd98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:50.386497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac027aa-936c-41f9-a6a8-d3850f01fd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:50.386779 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac027aa-936c-41f9-a6a8-d3850f01fd98 admin admin] PATH_INFO: `/v3/projects/b4244da044d6484b8d6eb365511af157/users/05bb1f831177435fbfe95348b79a1924/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:50.402804 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac027aa-936c-41f9-a6a8-d3850f01fd98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4244da044d6484b8d6eb365511af157, user_id=05bb1f831177435fbfe95348b79a1924, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:50.405958 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac027aa-936c-41f9-a6a8-d3850f01fd98 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:50.419304 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1322/2624] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:50 2022] PUT /identity/v3/projects/b4244da044d6484b8d6eb365511af157/users/05bb1f831177435fbfe95348b79a1924/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:50.430801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2257f31-86c5-43de-b270-8d915337b9a1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:50.431101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2257f31-86c5-43de-b270-8d915337b9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:50.431377 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2257f31-86c5-43de-b270-8d915337b9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:50.444427 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:50.462242 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c2257f31-86c5-43de-b270-8d915337b9a1 None None] MFA Rules not processed for user `05bb1f831177435fbfe95348b79a1924`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:50.463451 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c2257f31-86c5-43de-b270-8d915337b9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:50.517580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac027aa-936c-41f9-a6a8-d3850f01fd98 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:50.519240 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-eac027aa-936c-41f9-a6a8-d3850f01fd98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:50.530491 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1303/2625] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:50 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:50.534983 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2257f31-86c5-43de-b270-8d915337b9a1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:50.595741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac027aa-936c-41f9-a6a8-d3850f01fd98 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:50.598349 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2257f31-86c5-43de-b270-8d915337b9a1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:50.599416 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e705e-73b5-447e-b32e-c70ade144665 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:50.600068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e705e-73b5-447e-b32e-c70ade144665 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:50.600357 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e705e-73b5-447e-b32e-c70ade144665 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:50.600678 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e705e-73b5-447e-b32e-c70ade144665 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:50.602713 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-586e705e-73b5-447e-b32e-c70ade144665 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:50.603216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8963e-531e-4702-a0a0-7aac8634d27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:50.603872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8963e-531e-4702-a0a0-7aac8634d27b admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:50.604150 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8963e-531e-4702-a0a0-7aac8634d27b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:50.604430 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8963e-531e-4702-a0a0-7aac8634d27b admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:50.605146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8963e-531e-4702-a0a0-7aac8634d27b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:50.607035 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8963e-531e-4702-a0a0-7aac8634d27b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:50.620773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:50.633319 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1304/2626] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:36:50 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:36:50.637250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8963e-531e-4702-a0a0-7aac8634d27b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:50.664232 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e705e-73b5-447e-b32e-c70ade144665 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:50.665845 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e705e-73b5-447e-b32e-c70ade144665 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:50.675247 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1323/2627] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:50.699488 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8963e-531e-4702-a0a0-7aac8634d27b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:50.704366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585452d4-ef62-4c00-bf9f-3997d53d60c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:50.705069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585452d4-ef62-4c00-bf9f-3997d53d60c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:50.705363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585452d4-ef62-4c00-bf9f-3997d53d60c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:50.705648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585452d4-ef62-4c00-bf9f-3997d53d60c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:50.706344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585452d4-ef62-4c00-bf9f-3997d53d60c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:50.708302 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585452d4-ef62-4c00-bf9f-3997d53d60c8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:50.732406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1305/2628] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:36:50 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:36:50.733870 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e705e-73b5-447e-b32e-c70ade144665 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:50.745543 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e705e-73b5-447e-b32e-c70ade144665 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:50.750532 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dba413-39e0-46b0-a2ac-87ea0673b994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:50.751224 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dba413-39e0-46b0-a2ac-87ea0673b994 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:50.751540 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dba413-39e0-46b0-a2ac-87ea0673b994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:50.751822 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dba413-39e0-46b0-a2ac-87ea0673b994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:50.752571 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dba413-39e0-46b0-a2ac-87ea0673b994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:50.754197 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dba413-39e0-46b0-a2ac-87ea0673b994 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:50.765598 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1324/2629] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:50.771630 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585452d4-ef62-4c00-bf9f-3997d53d60c8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:50.782655 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585452d4-ef62-4c00-bf9f-3997d53d60c8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:50.788145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814aff1-3a31-462e-a86e-a07a14163ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:50.789098 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814aff1-3a31-462e-a86e-a07a14163ac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:50.789389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814aff1-3a31-462e-a86e-a07a14163ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:50.789667 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814aff1-3a31-462e-a86e-a07a14163ac6 admin admin] PATH_INFO: `/v3/projects/cf10ee298d0548a89765feabe29fbd12/users/125b0d56fbd04729a1d0cbc1e1fc543c/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:50.798933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:50.806975 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814aff1-3a31-462e-a86e-a07a14163ac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf10ee298d0548a89765feabe29fbd12, user_id=125b0d56fbd04729a1d0cbc1e1fc543c, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:50.809786 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814aff1-3a31-462e-a86e-a07a14163ac6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:50.823704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1306/2630] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:50 2022] PUT /identity/v3/projects/cf10ee298d0548a89765feabe29fbd12/users/125b0d56fbd04729a1d0cbc1e1fc543c/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:50.827812 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dba413-39e0-46b0-a2ac-87ea0673b994 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:50.838449 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dba413-39e0-46b0-a2ac-87ea0673b994 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:50.898831 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb74096-be55-4df3-953c-273297436c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:50.899505 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb74096-be55-4df3-953c-273297436c76 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:50.899793 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb74096-be55-4df3-953c-273297436c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:50.900070 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb74096-be55-4df3-953c-273297436c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:50.900834 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb74096-be55-4df3-953c-273297436c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:50.902444 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb74096-be55-4df3-953c-273297436c76 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:50.913989 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1325/2631] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:36:50.918499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814aff1-3a31-462e-a86e-a07a14163ac6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:50.930176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814aff1-3a31-462e-a86e-a07a14163ac6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:50.935004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0dee0-927d-45cc-ad1a-2d81c1325762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:50.935888 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0dee0-927d-45cc-ad1a-2d81c1325762 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:50.936172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0dee0-927d-45cc-ad1a-2d81c1325762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:50.936449 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0dee0-927d-45cc-ad1a-2d81c1325762 admin admin] PATH_INFO: `/v3/projects/cf10ee298d0548a89765feabe29fbd12/users/125b0d56fbd04729a1d0cbc1e1fc543c/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:50.938557 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0dee0-927d-45cc-ad1a-2d81c1325762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf10ee298d0548a89765feabe29fbd12, user_id=125b0d56fbd04729a1d0cbc1e1fc543c, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:50.941421 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0dee0-927d-45cc-ad1a-2d81c1325762 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:50.952281 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1307/2632] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:50 2022] PUT /identity/v3/projects/cf10ee298d0548a89765feabe29fbd12/users/125b0d56fbd04729a1d0cbc1e1fc543c/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:50.963279 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db464f-2d95-4382-9368-fa69098bcfa8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:50.963573 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db464f-2d95-4382-9368-fa69098bcfa8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:50.963853 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db464f-2d95-4382-9368-fa69098bcfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:50.994362 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-01db464f-2d95-4382-9368-fa69098bcfa8 None None] MFA Rules not processed for user `125b0d56fbd04729a1d0cbc1e1fc543c`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:50.995953 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-01db464f-2d95-4382-9368-fa69098bcfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:51.066965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1326/2633] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:50 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:51.083117 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0dee0-927d-45cc-ad1a-2d81c1325762 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:51.085459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bbe0dee0-927d-45cc-ad1a-2d81c1325762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:51.145085 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0dee0-927d-45cc-ad1a-2d81c1325762 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:51.149336 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aacdf1-2b1c-480a-9a9c-c584145c9ab5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:51.149992 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aacdf1-2b1c-480a-9a9c-c584145c9ab5 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:51.150397 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aacdf1-2b1c-480a-9a9c-c584145c9ab5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:51.150688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aacdf1-2b1c-480a-9a9c-c584145c9ab5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:51.160237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aacdf1-2b1c-480a-9a9c-c584145c9ab5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:51.162744 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aacdf1-2b1c-480a-9a9c-c584145c9ab5 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:51.173290 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1308/2634] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:51 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:51.195791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db464f-2d95-4382-9368-fa69098bcfa8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:51.205716 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db464f-2d95-4382-9368-fa69098bcfa8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:51.210386 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657e1cb-d43e-41c8-bcbd-350ffd0c9a34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:51.211172 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657e1cb-d43e-41c8-bcbd-350ffd0c9a34 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:51.211472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657e1cb-d43e-41c8-bcbd-350ffd0c9a34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:51.211754 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657e1cb-d43e-41c8-bcbd-350ffd0c9a34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:51.213693 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d657e1cb-d43e-41c8-bcbd-350ffd0c9a34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:51.242788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:51.245834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aacdf1-2b1c-480a-9a9c-c584145c9ab5 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:51.254396 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aacdf1-2b1c-480a-9a9c-c584145c9ab5 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:51.258644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785cd3b-29b2-48f3-a50d-b352ff71ad42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:51.259280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785cd3b-29b2-48f3-a50d-b352ff71ad42 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:51.259567 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785cd3b-29b2-48f3-a50d-b352ff71ad42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:51.259846 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785cd3b-29b2-48f3-a50d-b352ff71ad42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:51.282091 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657e1cb-d43e-41c8-bcbd-350ffd0c9a34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:51.283574 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657e1cb-d43e-41c8-bcbd-350ffd0c9a34 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:51.293089 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1327/2635] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:51.319748 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785cd3b-29b2-48f3-a50d-b352ff71ad42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:51.321227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785cd3b-29b2-48f3-a50d-b352ff71ad42 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:51.330545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1309/2636] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:51 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:54.072921 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657e1cb-d43e-41c8-bcbd-350ffd0c9a34 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:54.145075 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657e1cb-d43e-41c8-bcbd-350ffd0c9a34 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:54.155258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec7704-5f04-4bbd-a804-2f6c1f5631a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:54.156171 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec7704-5f04-4bbd-a804-2f6c1f5631a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:54.156456 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec7704-5f04-4bbd-a804-2f6c1f5631a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:54.156763 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec7704-5f04-4bbd-a804-2f6c1f5631a9 admin admin] PATH_INFO: `/v3/users/05bb1f831177435fbfe95348b79a1924` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:54.157985 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec7704-5f04-4bbd-a804-2f6c1f5631a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05bb1f831177435fbfe95348b79a1924)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:54.159899 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec7704-5f04-4bbd-a804-2f6c1f5631a9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:54.206750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-98ec7704-5f04-4bbd-a804-2f6c1f5631a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05bb1f831177435fbfe95348b79a1924'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:54.218468 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-98ec7704-5f04-4bbd-a804-2f6c1f5631a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05bb1f831177435fbfe95348b79a1924'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:54.219138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-98ec7704-5f04-4bbd-a804-2f6c1f5631a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05bb1f831177435fbfe95348b79a1924'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:54.219742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-98ec7704-5f04-4bbd-a804-2f6c1f5631a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05bb1f831177435fbfe95348b79a1924'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:54.228193 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-98ec7704-5f04-4bbd-a804-2f6c1f5631a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05bb1f831177435fbfe95348b79a1924'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:54.241579 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1328/2637] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:36:54 2022] DELETE /identity/v3/users/05bb1f831177435fbfe95348b79a1924 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:54.246287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bfb8d-b6f1-40b9-b1dc-66e16731d843 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:54.246771 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bfb8d-b6f1-40b9-b1dc-66e16731d843 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:54.246886 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bfb8d-b6f1-40b9-b1dc-66e16731d843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:54.263912 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-791bfb8d-b6f1-40b9-b1dc-66e16731d843 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:54.265133 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-791bfb8d-b6f1-40b9-b1dc-66e16731d843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:54.324753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1310/2638] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:54.335670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec7704-5f04-4bbd-a804-2f6c1f5631a9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:54.337092 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-98ec7704-5f04-4bbd-a804-2f6c1f5631a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:54.385480 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:54.399240 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec7704-5f04-4bbd-a804-2f6c1f5631a9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:54.402991 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065a4a9-4ff3-474f-a0be-e76629aaa031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:54.403641 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065a4a9-4ff3-474f-a0be-e76629aaa031 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:54.404107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065a4a9-4ff3-474f-a0be-e76629aaa031 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:54.404398 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065a4a9-4ff3-474f-a0be-e76629aaa031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:54.414044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065a4a9-4ff3-474f-a0be-e76629aaa031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:54.415571 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065a4a9-4ff3-474f-a0be-e76629aaa031 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:54.425190 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1329/2639] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:54.688591 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bfb8d-b6f1-40b9-b1dc-66e16731d843 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:54.690003 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-791bfb8d-b6f1-40b9-b1dc-66e16731d843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:54.703741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bfb8d-b6f1-40b9-b1dc-66e16731d843 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:54.709466 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a8731-e39b-43da-bfe2-2cb63baafe10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:54.710347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a8731-e39b-43da-bfe2-2cb63baafe10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:54.710704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a8731-e39b-43da-bfe2-2cb63baafe10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:54.710889 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a8731-e39b-43da-bfe2-2cb63baafe10 admin admin] PATH_INFO: `/v3/projects/b4244da044d6484b8d6eb365511af157` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:54.712174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a8731-e39b-43da-bfe2-2cb63baafe10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4244da044d6484b8d6eb365511af157)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:54.714102 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a8731-e39b-43da-bfe2-2cb63baafe10 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:54.725682 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:54.762652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fa9a8731-e39b-43da-bfe2-2cb63baafe10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4244da044d6484b8d6eb365511af157', 'request_id': 'req-fa9a8731-e39b-43da-bfe2-2cb63baafe10'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:54.768263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fa9a8731-e39b-43da-bfe2-2cb63baafe10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4244da044d6484b8d6eb365511af157', 'request_id': 'req-fa9a8731-e39b-43da-bfe2-2cb63baafe10'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:54.775209 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fa9a8731-e39b-43da-bfe2-2cb63baafe10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4244da044d6484b8d6eb365511af157', 'request_id': 'req-fa9a8731-e39b-43da-bfe2-2cb63baafe10'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:54.789159 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fa9a8731-e39b-43da-bfe2-2cb63baafe10 admin admin] The token cache is being invalidate because project b4244da044d6484b8d6eb365511af157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:36:54.789449 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fa9a8731-e39b-43da-bfe2-2cb63baafe10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:54.790070 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fa9a8731-e39b-43da-bfe2-2cb63baafe10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:54.791618 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1311/2640] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:36:54 2022] DELETE /identity/v3/projects/b4244da044d6484b8d6eb365511af157 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:54.795736 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065a4a9-4ff3-474f-a0be-e76629aaa031 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:54.805348 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065a4a9-4ff3-474f-a0be-e76629aaa031 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:54.860843 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ccc2e-1b8e-49dd-ac99-72eff9d48cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:54.861736 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ccc2e-1b8e-49dd-ac99-72eff9d48cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:54.862007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ccc2e-1b8e-49dd-ac99-72eff9d48cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:54.862285 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ccc2e-1b8e-49dd-ac99-72eff9d48cae admin admin] PATH_INFO: `/v3/users/125b0d56fbd04729a1d0cbc1e1fc543c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:54.863464 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ccc2e-1b8e-49dd-ac99-72eff9d48cae admin admin] RBAC: Authorizing `identity:delete_user(user_id=125b0d56fbd04729a1d0cbc1e1fc543c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:54.865419 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ccc2e-1b8e-49dd-ac99-72eff9d48cae admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:54.909714 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d78ccc2e-1b8e-49dd-ac99-72eff9d48cae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '125b0d56fbd04729a1d0cbc1e1fc543c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:54.922873 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d78ccc2e-1b8e-49dd-ac99-72eff9d48cae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '125b0d56fbd04729a1d0cbc1e1fc543c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:54.923540 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d78ccc2e-1b8e-49dd-ac99-72eff9d48cae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '125b0d56fbd04729a1d0cbc1e1fc543c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:54.924156 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d78ccc2e-1b8e-49dd-ac99-72eff9d48cae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '125b0d56fbd04729a1d0cbc1e1fc543c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:54.931554 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d78ccc2e-1b8e-49dd-ac99-72eff9d48cae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '125b0d56fbd04729a1d0cbc1e1fc543c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:54.944856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1330/2641] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:36:54 2022] DELETE /identity/v3/users/125b0d56fbd04729a1d0cbc1e1fc543c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:55.230400 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a8731-e39b-43da-bfe2-2cb63baafe10 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:55.233222 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ccc2e-1b8e-49dd-ac99-72eff9d48cae admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:55.233707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fa9a8731-e39b-43da-bfe2-2cb63baafe10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:55.234850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d78ccc2e-1b8e-49dd-ac99-72eff9d48cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:55.247149 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a8731-e39b-43da-bfe2-2cb63baafe10 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:55.304937 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c72945-40ad-4b8a-bab5-7b3b1b696a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:55.306074 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c72945-40ad-4b8a-bab5-7b3b1b696a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:55.306536 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c72945-40ad-4b8a-bab5-7b3b1b696a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:55.306749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c72945-40ad-4b8a-bab5-7b3b1b696a72 admin admin] PATH_INFO: `/v3/users/bcc941012f144612959498eafbae030c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:55.308121 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c72945-40ad-4b8a-bab5-7b3b1b696a72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcc941012f144612959498eafbae030c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:55.310345 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c72945-40ad-4b8a-bab5-7b3b1b696a72 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:55.314030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ccc2e-1b8e-49dd-ac99-72eff9d48cae admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:55.319877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d09f0-893a-416a-8b37-351e5ba16088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:55.320774 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d09f0-893a-416a-8b37-351e5ba16088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:55.321065 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d09f0-893a-416a-8b37-351e5ba16088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:55.321346 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d09f0-893a-416a-8b37-351e5ba16088 admin admin] PATH_INFO: `/v3/projects/cf10ee298d0548a89765feabe29fbd12` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:55.322577 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d09f0-893a-416a-8b37-351e5ba16088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf10ee298d0548a89765feabe29fbd12)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:55.324544 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d09f0-893a-416a-8b37-351e5ba16088 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:55.331063 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:55.360249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-36c72945-40ad-4b8a-bab5-7b3b1b696a72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcc941012f144612959498eafbae030c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:55.361019 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-36c72945-40ad-4b8a-bab5-7b3b1b696a72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcc941012f144612959498eafbae030c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:55.361730 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-36c72945-40ad-4b8a-bab5-7b3b1b696a72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcc941012f144612959498eafbae030c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:55.370253 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-36c72945-40ad-4b8a-bab5-7b3b1b696a72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcc941012f144612959498eafbae030c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:55.377986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eb2d09f0-893a-416a-8b37-351e5ba16088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf10ee298d0548a89765feabe29fbd12', 'request_id': 'req-eb2d09f0-893a-416a-8b37-351e5ba16088'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:55.385471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eb2d09f0-893a-416a-8b37-351e5ba16088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf10ee298d0548a89765feabe29fbd12', 'request_id': 'req-eb2d09f0-893a-416a-8b37-351e5ba16088'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:55.390925 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-36c72945-40ad-4b8a-bab5-7b3b1b696a72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcc941012f144612959498eafbae030c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:55.391957 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eb2d09f0-893a-416a-8b37-351e5ba16088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf10ee298d0548a89765feabe29fbd12', 'request_id': 'req-eb2d09f0-893a-416a-8b37-351e5ba16088'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:55.404947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1312/2642] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:36:55 2022] DELETE /identity/v3/users/bcc941012f144612959498eafbae030c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:55.406836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eb2d09f0-893a-416a-8b37-351e5ba16088 admin admin] The token cache is being invalidate because project cf10ee298d0548a89765feabe29fbd12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:36:55.407132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eb2d09f0-893a-416a-8b37-351e5ba16088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:55.407835 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eb2d09f0-893a-416a-8b37-351e5ba16088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:55.409399 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1331/2643] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:36:55 2022] DELETE /identity/v3/projects/cf10ee298d0548a89765feabe29fbd12 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:55.409556 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d1873-5d78-4898-b6e7-6f63ecbd115c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:55.409743 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d1873-5d78-4898-b6e7-6f63ecbd115c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:55.410013 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d1873-5d78-4898-b6e7-6f63ecbd115c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:55.415397 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c3a10-0d80-4f58-98d6-87198eb968be None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:55.415951 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c3a10-0d80-4f58-98d6-87198eb968be None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:55.416303 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c3a10-0d80-4f58-98d6-87198eb968be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:55.428056 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-016d1873-5d78-4898-b6e7-6f63ecbd115c None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:55.429295 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-016d1873-5d78-4898-b6e7-6f63ecbd115c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:55.436917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c53c3a10-0d80-4f58-98d6-87198eb968be None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:55.438007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c53c3a10-0d80-4f58-98d6-87198eb968be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:55.441037 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:55.497061 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1332/2644] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:55.501820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1313/2645] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:36:55.504662 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416550f9-76f2-4259-8410-c887fa9d88fd None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:55.504959 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416550f9-76f2-4259-8410-c887fa9d88fd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:55.505260 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416550f9-76f2-4259-8410-c887fa9d88fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:55.509337 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d1873-5d78-4898-b6e7-6f63ecbd115c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:55.510826 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-016d1873-5d78-4898-b6e7-6f63ecbd115c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:55.525065 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-416550f9-76f2-4259-8410-c887fa9d88fd None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:55.526660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-416550f9-76f2-4259-8410-c887fa9d88fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:55.535325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1333/2646] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:55.539742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416550f9-76f2-4259-8410-c887fa9d88fd None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:55.548891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416550f9-76f2-4259-8410-c887fa9d88fd None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:55.555229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3255a8ba-ec9c-4f6e-83f4-4806690d7b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:55.555891 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255a8ba-ec9c-4f6e-83f4-4806690d7b8c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:55.556178 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255a8ba-ec9c-4f6e-83f4-4806690d7b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:55.556456 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3255a8ba-ec9c-4f6e-83f4-4806690d7b8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:55.557242 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3255a8ba-ec9c-4f6e-83f4-4806690d7b8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:36:55.557547 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3255a8ba-ec9c-4f6e-83f4-4806690d7b8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:55.559135 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3255a8ba-ec9c-4f6e-83f4-4806690d7b8c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:55.569099 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d1873-5d78-4898-b6e7-6f63ecbd115c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:55.571460 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1334/2647] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:36:55.572911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58776b04-511b-4b0f-ba85-651c1eaafdf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:55.573672 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58776b04-511b-4b0f-ba85-651c1eaafdf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:55.573960 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58776b04-511b-4b0f-ba85-651c1eaafdf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:55.574237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58776b04-511b-4b0f-ba85-651c1eaafdf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:55.575955 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3255a8ba-ec9c-4f6e-83f4-4806690d7b8c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:55.583747 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58776b04-511b-4b0f-ba85-651c1eaafdf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:55.585208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58776b04-511b-4b0f-ba85-651c1eaafdf6 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:55.585750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3255a8ba-ec9c-4f6e-83f4-4806690d7b8c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:55.590553 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6350df0-6ca2-4f6f-a5f6-c4f4ca46b186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:55.591181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6350df0-6ca2-4f6f-a5f6-c4f4ca46b186 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:55.591466 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6350df0-6ca2-4f6f-a5f6-c4f4ca46b186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:55.591742 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6350df0-6ca2-4f6f-a5f6-c4f4ca46b186 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:55.592454 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6350df0-6ca2-4f6f-a5f6-c4f4ca46b186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:55.594371 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6350df0-6ca2-4f6f-a5f6-c4f4ca46b186 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:55.595437 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1314/2648] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:36:55.622461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1335/2649] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:36:55 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:36:55.626793 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58776b04-511b-4b0f-ba85-651c1eaafdf6 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:55.637802 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58776b04-511b-4b0f-ba85-651c1eaafdf6 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:55.698652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c3f5b-b9fb-46fa-98bc-659c158c6d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:55.699383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c3f5b-b9fb-46fa-98bc-659c158c6d1d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:55.699673 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c3f5b-b9fb-46fa-98bc-659c158c6d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:55.699947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c3f5b-b9fb-46fa-98bc-659c158c6d1d admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:55.700680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c3f5b-b9fb-46fa-98bc-659c158c6d1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:55.702654 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c3f5b-b9fb-46fa-98bc-659c158c6d1d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:55.726843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1315/2650] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:36:55 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:55.728201 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6350df0-6ca2-4f6f-a5f6-c4f4ca46b186 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:55.742018 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6350df0-6ca2-4f6f-a5f6-c4f4ca46b186 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:55.747128 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e33f2-a744-4d53-8c95-4bf18e68fd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:55.747884 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e33f2-a744-4d53-8c95-4bf18e68fd7b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:55.748179 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e33f2-a744-4d53-8c95-4bf18e68fd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:55.748457 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e33f2-a744-4d53-8c95-4bf18e68fd7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:55.749248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e33f2-a744-4d53-8c95-4bf18e68fd7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:55.750865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e33f2-a744-4d53-8c95-4bf18e68fd7b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:55.762544 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1336/2651] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:55.766604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c3f5b-b9fb-46fa-98bc-659c158c6d1d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:55.778417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c3f5b-b9fb-46fa-98bc-659c158c6d1d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:55.783140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35399d9a-e4bd-409c-8aa4-57034b592ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:55.783806 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35399d9a-e4bd-409c-8aa4-57034b592ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:55.784091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35399d9a-e4bd-409c-8aa4-57034b592ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:55.784373 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35399d9a-e4bd-409c-8aa4-57034b592ff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:55.785148 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35399d9a-e4bd-409c-8aa4-57034b592ff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:55.787313 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35399d9a-e4bd-409c-8aa4-57034b592ff6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:55.797636 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1316/2652] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:55.801697 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e33f2-a744-4d53-8c95-4bf18e68fd7b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:55.812263 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e33f2-a744-4d53-8c95-4bf18e68fd7b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:55.817086 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb683a3-951d-44c9-920c-260e8d62df06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:55.818014 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb683a3-951d-44c9-920c-260e8d62df06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:55.818304 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb683a3-951d-44c9-920c-260e8d62df06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:55.818580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb683a3-951d-44c9-920c-260e8d62df06 admin admin] PATH_INFO: `/v3/projects/eb9fa727a0ff483497ce25bef22b036f/users/a6fa0a6ff4d44288a46b49607e85a880/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:55.834638 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb683a3-951d-44c9-920c-260e8d62df06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb9fa727a0ff483497ce25bef22b036f, user_id=a6fa0a6ff4d44288a46b49607e85a880, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:55.837355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb683a3-951d-44c9-920c-260e8d62df06 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:55.851292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1337/2653] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:55 2022] PUT /identity/v3/projects/eb9fa727a0ff483497ce25bef22b036f/users/a6fa0a6ff4d44288a46b49607e85a880/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:55.862876 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ec694-981d-4aa7-a440-35e02dd440e6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:55.863175 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ec694-981d-4aa7-a440-35e02dd440e6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:55.863618 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ec694-981d-4aa7-a440-35e02dd440e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:55.884311 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb683a3-951d-44c9-920c-260e8d62df06 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:55.886426 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6cb683a3-951d-44c9-920c-260e8d62df06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:55.898463 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ad7ec694-981d-4aa7-a440-35e02dd440e6 None None] MFA Rules not processed for user `a6fa0a6ff4d44288a46b49607e85a880`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:55.899711 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ad7ec694-981d-4aa7-a440-35e02dd440e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:55.910676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:55.940418 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:55.959765 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb683a3-951d-44c9-920c-260e8d62df06 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:55.964814 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1a56b-667a-4eff-a224-026e3ad00dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:55.965671 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1a56b-667a-4eff-a224-026e3ad00dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:55.965952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1a56b-667a-4eff-a224-026e3ad00dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:55.966229 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1a56b-667a-4eff-a224-026e3ad00dfb admin admin] PATH_INFO: `/v3/projects/86e658ce6d174f678a9e7a67da6129ad` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:55.967445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1a56b-667a-4eff-a224-026e3ad00dfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=86e658ce6d174f678a9e7a67da6129ad)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:55.969377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1a56b-667a-4eff-a224-026e3ad00dfb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:55.979572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1317/2654] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:55.983901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ec694-981d-4aa7-a440-35e02dd440e6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:55.994746 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ec694-981d-4aa7-a440-35e02dd440e6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:55.999853 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3772f45-92e3-4849-abe0-f581cf31d294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:56.000609 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3772f45-92e3-4849-abe0-f581cf31d294 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:56.000903 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3772f45-92e3-4849-abe0-f581cf31d294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:56.001187 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3772f45-92e3-4849-abe0-f581cf31d294 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:56.001875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3772f45-92e3-4849-abe0-f581cf31d294 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:56.003833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3772f45-92e3-4849-abe0-f581cf31d294 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:56.017403 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-48c1a56b-667a-4eff-a224-026e3ad00dfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86e658ce6d174f678a9e7a67da6129ad', 'request_id': 'req-48c1a56b-667a-4eff-a224-026e3ad00dfb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.023945 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-48c1a56b-667a-4eff-a224-026e3ad00dfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86e658ce6d174f678a9e7a67da6129ad', 'request_id': 'req-48c1a56b-667a-4eff-a224-026e3ad00dfb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.031645 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1318/2655] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:36:55 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:36:56.031819 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-48c1a56b-667a-4eff-a224-026e3ad00dfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86e658ce6d174f678a9e7a67da6129ad', 'request_id': 'req-48c1a56b-667a-4eff-a224-026e3ad00dfb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.032880 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3772f45-92e3-4849-abe0-f581cf31d294 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:56.044095 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-48c1a56b-667a-4eff-a224-026e3ad00dfb admin admin] The token cache is being invalidate because project 86e658ce6d174f678a9e7a67da6129ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:36:56.044388 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-48c1a56b-667a-4eff-a224-026e3ad00dfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.045058 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-48c1a56b-667a-4eff-a224-026e3ad00dfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.046607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1338/2656] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:36:55 2022] DELETE /identity/v3/projects/86e658ce6d174f678a9e7a67da6129ad => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:56.050470 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1a56b-667a-4eff-a224-026e3ad00dfb admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:56.051785 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-48c1a56b-667a-4eff-a224-026e3ad00dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:56.097167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3772f45-92e3-4849-abe0-f581cf31d294 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:56.103558 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79ba58-f04e-41ec-b23e-ce394abd400f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:56.104314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79ba58-f04e-41ec-b23e-ce394abd400f admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:56.104641 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79ba58-f04e-41ec-b23e-ce394abd400f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:56.104932 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79ba58-f04e-41ec-b23e-ce394abd400f admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:56.105631 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79ba58-f04e-41ec-b23e-ce394abd400f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:56.107633 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79ba58-f04e-41ec-b23e-ce394abd400f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:56.118644 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1a56b-667a-4eff-a224-026e3ad00dfb admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:56.123697 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554781a-247f-417a-9b38-9af89a665f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:56.124602 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554781a-247f-417a-9b38-9af89a665f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:56.124889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554781a-247f-417a-9b38-9af89a665f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:56.125194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554781a-247f-417a-9b38-9af89a665f18 admin admin] PATH_INFO: `/v3/users/4c480c73b3d748959c7f206325de2b8e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:56.126408 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554781a-247f-417a-9b38-9af89a665f18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c480c73b3d748959c7f206325de2b8e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:56.128344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554781a-247f-417a-9b38-9af89a665f18 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:56.129082 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1319/2657] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:36:56 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:36:56.133004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79ba58-f04e-41ec-b23e-ce394abd400f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:56.134651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0b79ba58-f04e-41ec-b23e-ce394abd400f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:56.148860 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79ba58-f04e-41ec-b23e-ce394abd400f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:56.154632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce927d3-4919-43c0-ad3a-21e3ad139aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:56.155599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce927d3-4919-43c0-ad3a-21e3ad139aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:56.156192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce927d3-4919-43c0-ad3a-21e3ad139aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:56.156695 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce927d3-4919-43c0-ad3a-21e3ad139aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:56.157602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce927d3-4919-43c0-ad3a-21e3ad139aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:56.159585 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce927d3-4919-43c0-ad3a-21e3ad139aa5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:56.172589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1320/2658] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:36:56.176908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce927d3-4919-43c0-ad3a-21e3ad139aa5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:56.190631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d554781a-247f-417a-9b38-9af89a665f18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c480c73b3d748959c7f206325de2b8e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.191503 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce927d3-4919-43c0-ad3a-21e3ad139aa5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:56.193094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:56.198461 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780a941-5f19-4f18-91cb-011bc79d0de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:56.199497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780a941-5f19-4f18-91cb-011bc79d0de3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:56.199871 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780a941-5f19-4f18-91cb-011bc79d0de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:56.200221 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780a941-5f19-4f18-91cb-011bc79d0de3 admin admin] PATH_INFO: `/v3/projects/4c09d709e42448e09e0d8f82ba6372b9/users/251f5164caa84695a4b82a650abee0b5/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:56.211703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d554781a-247f-417a-9b38-9af89a665f18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c480c73b3d748959c7f206325de2b8e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.212499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d554781a-247f-417a-9b38-9af89a665f18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c480c73b3d748959c7f206325de2b8e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.213256 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d554781a-247f-417a-9b38-9af89a665f18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c480c73b3d748959c7f206325de2b8e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.221768 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e780a941-5f19-4f18-91cb-011bc79d0de3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c09d709e42448e09e0d8f82ba6372b9, user_id=251f5164caa84695a4b82a650abee0b5, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:56.225175 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e780a941-5f19-4f18-91cb-011bc79d0de3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:56.229153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d554781a-247f-417a-9b38-9af89a665f18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c480c73b3d748959c7f206325de2b8e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.240855 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1321/2659] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:56 2022] PUT /identity/v3/projects/4c09d709e42448e09e0d8f82ba6372b9/users/251f5164caa84695a4b82a650abee0b5/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:56.244237 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1339/2660] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:36:56 2022] DELETE /identity/v3/users/4c480c73b3d748959c7f206325de2b8e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:56.244810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780a941-5f19-4f18-91cb-011bc79d0de3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:56.246405 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e780a941-5f19-4f18-91cb-011bc79d0de3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:56.328833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780a941-5f19-4f18-91cb-011bc79d0de3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:56.335970 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:56.419858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51dd00a-a19e-4544-a447-27d9ac3afdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:56.420601 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51dd00a-a19e-4544-a447-27d9ac3afdf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:56.420911 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51dd00a-a19e-4544-a447-27d9ac3afdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:56.421164 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51dd00a-a19e-4544-a447-27d9ac3afdf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:56.421921 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51dd00a-a19e-4544-a447-27d9ac3afdf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:56.423534 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51dd00a-a19e-4544-a447-27d9ac3afdf9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:56.436146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1322/2661] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:36:56.440359 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554781a-247f-417a-9b38-9af89a665f18 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:56.452989 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554781a-247f-417a-9b38-9af89a665f18 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:56.468954 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b0381-04c5-4d3c-b53b-afc050b3ed2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:56.470114 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b0381-04c5-4d3c-b53b-afc050b3ed2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:56.472471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b0381-04c5-4d3c-b53b-afc050b3ed2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:56.473064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b0381-04c5-4d3c-b53b-afc050b3ed2b admin admin] PATH_INFO: `/v3/projects/4c09d709e42448e09e0d8f82ba6372b9/users/251f5164caa84695a4b82a650abee0b5/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:56.475944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b0381-04c5-4d3c-b53b-afc050b3ed2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c09d709e42448e09e0d8f82ba6372b9, user_id=251f5164caa84695a4b82a650abee0b5, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:56.478977 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b0381-04c5-4d3c-b53b-afc050b3ed2b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:56.492376 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1340/2662] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:56 2022] PUT /identity/v3/projects/4c09d709e42448e09e0d8f82ba6372b9/users/251f5164caa84695a4b82a650abee0b5/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:56.502404 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f286ede-df77-47ef-8fae-ecaa93680234 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:56.502749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f286ede-df77-47ef-8fae-ecaa93680234 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:56.502954 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f286ede-df77-47ef-8fae-ecaa93680234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:56.532804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9f286ede-df77-47ef-8fae-ecaa93680234 None None] MFA Rules not processed for user `251f5164caa84695a4b82a650abee0b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:56.534036 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9f286ede-df77-47ef-8fae-ecaa93680234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:56.556613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b0381-04c5-4d3c-b53b-afc050b3ed2b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:56.558444 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-de6b0381-04c5-4d3c-b53b-afc050b3ed2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:56.614192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1323/2663] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:56.625607 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f286ede-df77-47ef-8fae-ecaa93680234 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:56.626948 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9f286ede-df77-47ef-8fae-ecaa93680234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:56.634587 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b0381-04c5-4d3c-b53b-afc050b3ed2b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:56.642225 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9a0d4-af77-406b-887d-36bd3e2b842d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:56.644329 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9a0d4-af77-406b-887d-36bd3e2b842d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:56.644655 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9a0d4-af77-406b-887d-36bd3e2b842d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:56.644941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9a0d4-af77-406b-887d-36bd3e2b842d admin admin] PATH_INFO: `/v3/projects/dcb37bf146a348c2acd4a998f24f6ddb` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:56.646745 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9a0d4-af77-406b-887d-36bd3e2b842d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcb37bf146a348c2acd4a998f24f6ddb)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:56.648965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9a0d4-af77-406b-887d-36bd3e2b842d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:56.696465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f286ede-df77-47ef-8fae-ecaa93680234 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:56.696841 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d5c9a0d4-af77-406b-887d-36bd3e2b842d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcb37bf146a348c2acd4a998f24f6ddb', 'request_id': 'req-d5c9a0d4-af77-406b-887d-36bd3e2b842d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.702556 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d5c9a0d4-af77-406b-887d-36bd3e2b842d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcb37bf146a348c2acd4a998f24f6ddb', 'request_id': 'req-d5c9a0d4-af77-406b-887d-36bd3e2b842d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.709453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d5c9a0d4-af77-406b-887d-36bd3e2b842d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcb37bf146a348c2acd4a998f24f6ddb', 'request_id': 'req-d5c9a0d4-af77-406b-887d-36bd3e2b842d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.724392 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d5c9a0d4-af77-406b-887d-36bd3e2b842d admin admin] The token cache is being invalidate because project dcb37bf146a348c2acd4a998f24f6ddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:36:56.724833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d5c9a0d4-af77-406b-887d-36bd3e2b842d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.725618 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d5c9a0d4-af77-406b-887d-36bd3e2b842d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.727445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1341/2664] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:36:56 2022] DELETE /identity/v3/projects/dcb37bf146a348c2acd4a998f24f6ddb => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:56.731247 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9a0d4-af77-406b-887d-36bd3e2b842d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:56.732567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d5c9a0d4-af77-406b-887d-36bd3e2b842d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:56.742727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:56.748301 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55071ad-cf85-469f-8f67-0d3bc85812a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:56.749010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55071ad-cf85-469f-8f67-0d3bc85812a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:56.749297 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55071ad-cf85-469f-8f67-0d3bc85812a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:56.749574 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55071ad-cf85-469f-8f67-0d3bc85812a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:56.751342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d55071ad-cf85-469f-8f67-0d3bc85812a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:56.783290 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:56.803217 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9a0d4-af77-406b-887d-36bd3e2b842d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:56.808381 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4243c-8e76-47dd-bc39-5762c701c12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:56.809306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4243c-8e76-47dd-bc39-5762c701c12a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:56.809591 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4243c-8e76-47dd-bc39-5762c701c12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:56.809870 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4243c-8e76-47dd-bc39-5762c701c12a admin admin] PATH_INFO: `/v3/users/1990f1d45aa841e69f0c6d65011a9be3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:56.811072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e4243c-8e76-47dd-bc39-5762c701c12a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1990f1d45aa841e69f0c6d65011a9be3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:56.813046 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e4243c-8e76-47dd-bc39-5762c701c12a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:56.818388 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55071ad-cf85-469f-8f67-0d3bc85812a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:56.819877 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55071ad-cf85-469f-8f67-0d3bc85812a0 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:56.829831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1324/2665] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:36:56.862293 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-27e4243c-8e76-47dd-bc39-5762c701c12a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1990f1d45aa841e69f0c6d65011a9be3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.878123 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-27e4243c-8e76-47dd-bc39-5762c701c12a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1990f1d45aa841e69f0c6d65011a9be3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.878863 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-27e4243c-8e76-47dd-bc39-5762c701c12a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1990f1d45aa841e69f0c6d65011a9be3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.879496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-27e4243c-8e76-47dd-bc39-5762c701c12a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1990f1d45aa841e69f0c6d65011a9be3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.887978 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-27e4243c-8e76-47dd-bc39-5762c701c12a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1990f1d45aa841e69f0c6d65011a9be3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:56.898268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55071ad-cf85-469f-8f67-0d3bc85812a0 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:56.899807 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d55071ad-cf85-469f-8f67-0d3bc85812a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:56.902107 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1342/2666] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:36:56 2022] DELETE /identity/v3/users/1990f1d45aa841e69f0c6d65011a9be3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:56.914470 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55071ad-cf85-469f-8f67-0d3bc85812a0 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:56.977519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80aea6-d451-419c-9d1a-84f0bf7f9922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:56.978327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80aea6-d451-419c-9d1a-84f0bf7f9922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:56.978608 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80aea6-d451-419c-9d1a-84f0bf7f9922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:56.978886 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80aea6-d451-419c-9d1a-84f0bf7f9922 admin admin] PATH_INFO: `/v3/users/a6fa0a6ff4d44288a46b49607e85a880` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:56.980120 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80aea6-d451-419c-9d1a-84f0bf7f9922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6fa0a6ff4d44288a46b49607e85a880)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:56.982119 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80aea6-d451-419c-9d1a-84f0bf7f9922 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:57.033697 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2d80aea6-d451-419c-9d1a-84f0bf7f9922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6fa0a6ff4d44288a46b49607e85a880'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:57.034380 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2d80aea6-d451-419c-9d1a-84f0bf7f9922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6fa0a6ff4d44288a46b49607e85a880'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:57.034997 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2d80aea6-d451-419c-9d1a-84f0bf7f9922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6fa0a6ff4d44288a46b49607e85a880'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:57.043057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2d80aea6-d451-419c-9d1a-84f0bf7f9922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6fa0a6ff4d44288a46b49607e85a880'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:57.059914 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2d80aea6-d451-419c-9d1a-84f0bf7f9922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6fa0a6ff4d44288a46b49607e85a880'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:57.077259 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1325/2667] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:36:56 2022] DELETE /identity/v3/users/a6fa0a6ff4d44288a46b49607e85a880 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:57.082222 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad008d-f415-43eb-88c6-4beb60fd98bd None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:57.082507 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad008d-f415-43eb-88c6-4beb60fd98bd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:57.082834 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad008d-f415-43eb-88c6-4beb60fd98bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:57.105529 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c6ad008d-f415-43eb-88c6-4beb60fd98bd None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:57.106735 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c6ad008d-f415-43eb-88c6-4beb60fd98bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:57.180920 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1343/2668] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:57.185789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80aea6-d451-419c-9d1a-84f0bf7f9922 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:57.187180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2d80aea6-d451-419c-9d1a-84f0bf7f9922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:57.193841 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad008d-f415-43eb-88c6-4beb60fd98bd None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:57.195580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c6ad008d-f415-43eb-88c6-4beb60fd98bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:57.208071 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad008d-f415-43eb-88c6-4beb60fd98bd None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:57.213651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a66eb6-d4ad-4e67-ab8e-657aa5652a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:57.214642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a66eb6-d4ad-4e67-ab8e-657aa5652a12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:57.214936 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a66eb6-d4ad-4e67-ab8e-657aa5652a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:57.215231 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a66eb6-d4ad-4e67-ab8e-657aa5652a12 admin admin] PATH_INFO: `/v3/projects/f3d77150222e41778be09a33b20a905b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:57.216426 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a66eb6-d4ad-4e67-ab8e-657aa5652a12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3d77150222e41778be09a33b20a905b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:57.218374 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a66eb6-d4ad-4e67-ab8e-657aa5652a12 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:57.229612 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:57.266710 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-68a66eb6-d4ad-4e67-ab8e-657aa5652a12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3d77150222e41778be09a33b20a905b', 'request_id': 'req-68a66eb6-d4ad-4e67-ab8e-657aa5652a12'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:57.272269 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80aea6-d451-419c-9d1a-84f0bf7f9922 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:57.273471 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-68a66eb6-d4ad-4e67-ab8e-657aa5652a12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3d77150222e41778be09a33b20a905b', 'request_id': 'req-68a66eb6-d4ad-4e67-ab8e-657aa5652a12'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:57.274625 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:57.280999 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-68a66eb6-d4ad-4e67-ab8e-657aa5652a12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3d77150222e41778be09a33b20a905b', 'request_id': 'req-68a66eb6-d4ad-4e67-ab8e-657aa5652a12'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:57.296584 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-68a66eb6-d4ad-4e67-ab8e-657aa5652a12 admin admin] The token cache is being invalidate because project f3d77150222e41778be09a33b20a905b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:36:57.296874 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-68a66eb6-d4ad-4e67-ab8e-657aa5652a12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:57.297518 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-68a66eb6-d4ad-4e67-ab8e-657aa5652a12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:57.299079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1344/2669] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:36:57 2022] DELETE /identity/v3/projects/f3d77150222e41778be09a33b20a905b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:57.304538 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b9c63-a823-4687-9e9a-c5b3e9ed2a96 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:57.304853 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b9c63-a823-4687-9e9a-c5b3e9ed2a96 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:57.305179 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b9c63-a823-4687-9e9a-c5b3e9ed2a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:57.326833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fa4b9c63-a823-4687-9e9a-c5b3e9ed2a96 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:57.328310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fa4b9c63-a823-4687-9e9a-c5b3e9ed2a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:57.334741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393008f-fb3d-4f65-89df-5624866dc9bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:57.335553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393008f-fb3d-4f65-89df-5624866dc9bc service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:57.336599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393008f-fb3d-4f65-89df-5624866dc9bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:57.336904 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393008f-fb3d-4f65-89df-5624866dc9bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:57.338960 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4393008f-fb3d-4f65-89df-5624866dc9bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:57.401485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1345/2670] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:36:57.408951 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5730a6-63a7-4e06-a944-55a42e320d9b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:57.409277 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5730a6-63a7-4e06-a944-55a42e320d9b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:57.410278 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5730a6-63a7-4e06-a944-55a42e320d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:57.413953 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4393008f-fb3d-4f65-89df-5624866dc9bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:57.415411 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4393008f-fb3d-4f65-89df-5624866dc9bc service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:57.425297 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1326/2671] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:57.429392 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2c5730a6-63a7-4e06-a944-55a42e320d9b None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:57.430594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2c5730a6-63a7-4e06-a944-55a42e320d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:57.438670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1346/2672] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:57.443574 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393008f-fb3d-4f65-89df-5624866dc9bc service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:57.453363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393008f-fb3d-4f65-89df-5624866dc9bc service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:57.457999 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309ebe7-1573-4687-9f96-a71d19770079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:57.459322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309ebe7-1573-4687-9f96-a71d19770079 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:57.459734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309ebe7-1573-4687-9f96-a71d19770079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:57.459954 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309ebe7-1573-4687-9f96-a71d19770079 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:57.460660 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309ebe7-1573-4687-9f96-a71d19770079 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:36:57.460970 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309ebe7-1573-4687-9f96-a71d19770079 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:57.462832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309ebe7-1573-4687-9f96-a71d19770079 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:57.476971 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1327/2673] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:36:57.481908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5730a6-63a7-4e06-a944-55a42e320d9b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:57.491593 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5730a6-63a7-4e06-a944-55a42e320d9b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:57.496779 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f577e62-7e06-44f1-803b-860980d4c480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:57.497499 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f577e62-7e06-44f1-803b-860980d4c480 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:57.497779 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f577e62-7e06-44f1-803b-860980d4c480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:57.498099 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f577e62-7e06-44f1-803b-860980d4c480 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:57.498944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f577e62-7e06-44f1-803b-860980d4c480 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:57.500879 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f577e62-7e06-44f1-803b-860980d4c480 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:57.536997 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1347/2674] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:36:57 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:57.541596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309ebe7-1573-4687-9f96-a71d19770079 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:57.552162 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309ebe7-1573-4687-9f96-a71d19770079 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:57.625143 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6bad07-28cb-4134-b547-317c59120141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:57.625823 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6bad07-28cb-4134-b547-317c59120141 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:57.626101 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6bad07-28cb-4134-b547-317c59120141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:57.626377 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6bad07-28cb-4134-b547-317c59120141 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:57.627069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6bad07-28cb-4134-b547-317c59120141 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:57.629031 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6bad07-28cb-4134-b547-317c59120141 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:57.655451 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:57.661860 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1328/2675] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:36:57 2022] POST /identity/v3/users => generated 286 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:57.666227 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f577e62-7e06-44f1-803b-860980d4c480 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:57.680577 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f577e62-7e06-44f1-803b-860980d4c480 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:57.685834 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdae303-df46-4a3e-ae5d-b34dc8126342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:57.686608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdae303-df46-4a3e-ae5d-b34dc8126342 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:57.686871 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdae303-df46-4a3e-ae5d-b34dc8126342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:57.687717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdae303-df46-4a3e-ae5d-b34dc8126342 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:57.688106 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdae303-df46-4a3e-ae5d-b34dc8126342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:57.689836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdae303-df46-4a3e-ae5d-b34dc8126342 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:57.705006 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1348/2676] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:57.706664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6bad07-28cb-4134-b547-317c59120141 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:57.719155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6bad07-28cb-4134-b547-317c59120141 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:57.725547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e843ec-7fb7-4731-856d-2ad200b58f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:57.727347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e843ec-7fb7-4731-856d-2ad200b58f78 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:57.728446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e843ec-7fb7-4731-856d-2ad200b58f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:57.728446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e843ec-7fb7-4731-856d-2ad200b58f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:57.729098 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e843ec-7fb7-4731-856d-2ad200b58f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:57.731249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e843ec-7fb7-4731-856d-2ad200b58f78 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:57.745774 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1329/2677] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:57.759815 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdae303-df46-4a3e-ae5d-b34dc8126342 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:57.772252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e843ec-7fb7-4731-856d-2ad200b58f78 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:57.774804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-04e843ec-7fb7-4731-856d-2ad200b58f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:57.776987 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdae303-df46-4a3e-ae5d-b34dc8126342 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:57.783392 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e3cd0-5bcd-495f-b6ed-7b4d9eb31ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:57.785440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e3cd0-5bcd-495f-b6ed-7b4d9eb31ecc admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:57.786141 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e3cd0-5bcd-495f-b6ed-7b4d9eb31ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:57.786741 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e3cd0-5bcd-495f-b6ed-7b4d9eb31ecc admin admin] PATH_INFO: `/v3/projects/79d1632d8aa441b782effcc0008b9631/users/199f0ee6813f484ca3a390c2bc4600fd/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:57.796485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e843ec-7fb7-4731-856d-2ad200b58f78 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:57.808248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e3cd0-5bcd-495f-b6ed-7b4d9eb31ecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d1632d8aa441b782effcc0008b9631, user_id=199f0ee6813f484ca3a390c2bc4600fd, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:57.811536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e3cd0-5bcd-495f-b6ed-7b4d9eb31ecc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:57.816777 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b92e9-baaf-422e-bda0-87aa38828874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:57.818569 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b92e9-baaf-422e-bda0-87aa38828874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:57.818844 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b92e9-baaf-422e-bda0-87aa38828874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:57.819128 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b92e9-baaf-422e-bda0-87aa38828874 admin admin] PATH_INFO: `/v3/projects/eb9fa727a0ff483497ce25bef22b036f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:57.820632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b92e9-baaf-422e-bda0-87aa38828874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb9fa727a0ff483497ce25bef22b036f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:57.822438 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b92e9-baaf-422e-bda0-87aa38828874 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:57.836202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1349/2678] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:57 2022] PUT /identity/v3/projects/79d1632d8aa441b782effcc0008b9631/users/199f0ee6813f484ca3a390c2bc4600fd/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:57.846828 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92213896-7f54-4189-9eef-f8b7bab53ecf None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:57.847139 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92213896-7f54-4189-9eef-f8b7bab53ecf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:57.847615 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92213896-7f54-4189-9eef-f8b7bab53ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:57.879748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-92213896-7f54-4189-9eef-f8b7bab53ecf None None] MFA Rules not processed for user `199f0ee6813f484ca3a390c2bc4600fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:57.881584 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-92213896-7f54-4189-9eef-f8b7bab53ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:57.883639 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f45b92e9-baaf-422e-bda0-87aa38828874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb9fa727a0ff483497ce25bef22b036f', 'request_id': 'req-f45b92e9-baaf-422e-bda0-87aa38828874'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:57.890991 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f45b92e9-baaf-422e-bda0-87aa38828874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb9fa727a0ff483497ce25bef22b036f', 'request_id': 'req-f45b92e9-baaf-422e-bda0-87aa38828874'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:57.898281 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f45b92e9-baaf-422e-bda0-87aa38828874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb9fa727a0ff483497ce25bef22b036f', 'request_id': 'req-f45b92e9-baaf-422e-bda0-87aa38828874'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:57.916863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f45b92e9-baaf-422e-bda0-87aa38828874 admin admin] The token cache is being invalidate because project eb9fa727a0ff483497ce25bef22b036f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:36:57.917194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f45b92e9-baaf-422e-bda0-87aa38828874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:57.917908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f45b92e9-baaf-422e-bda0-87aa38828874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:57.920619 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1330/2679] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:36:57 2022] DELETE /identity/v3/projects/eb9fa727a0ff483497ce25bef22b036f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:57.924939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b92e9-baaf-422e-bda0-87aa38828874 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:57.926296 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f45b92e9-baaf-422e-bda0-87aa38828874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:57.959847 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1350/2680] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:57.978589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92213896-7f54-4189-9eef-f8b7bab53ecf None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:57.980638 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-92213896-7f54-4189-9eef-f8b7bab53ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:57.998234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b92e9-baaf-422e-bda0-87aa38828874 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:57.998952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:58.003454 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce40e21-a13e-4639-9d74-8b516256f2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:58.004410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce40e21-a13e-4639-9d74-8b516256f2ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:58.004778 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce40e21-a13e-4639-9d74-8b516256f2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:58.005232 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce40e21-a13e-4639-9d74-8b516256f2ff admin admin] PATH_INFO: `/v3/users/251f5164caa84695a4b82a650abee0b5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:58.006494 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce40e21-a13e-4639-9d74-8b516256f2ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=251f5164caa84695a4b82a650abee0b5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:58.008906 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce40e21-a13e-4639-9d74-8b516256f2ff admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:58.047466 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92213896-7f54-4189-9eef-f8b7bab53ecf None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:58.051203 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f9d09-90a0-45af-bd20-3ce27ff48482 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cee4d23797e430e9eeddc394ed7f50e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:58.051854 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f9d09-90a0-45af-bd20-3ce27ff48482 service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:58.052900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f9d09-90a0-45af-bd20-3ce27ff48482 service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:58.053191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f9d09-90a0-45af-bd20-3ce27ff48482 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:58.056843 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7ce40e21-a13e-4639-9d74-8b516256f2ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '251f5164caa84695a4b82a650abee0b5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:58.057779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7ce40e21-a13e-4639-9d74-8b516256f2ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '251f5164caa84695a4b82a650abee0b5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:58.058458 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7ce40e21-a13e-4639-9d74-8b516256f2ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '251f5164caa84695a4b82a650abee0b5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:58.061979 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f9d09-90a0-45af-bd20-3ce27ff48482 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:58.063748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f9d09-90a0-45af-bd20-3ce27ff48482 service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:58.066304 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7ce40e21-a13e-4639-9d74-8b516256f2ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '251f5164caa84695a4b82a650abee0b5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:58.073205 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1351/2681] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:36:58.080234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:58.081643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7ce40e21-a13e-4639-9d74-8b516256f2ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '251f5164caa84695a4b82a650abee0b5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:58.099487 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1331/2682] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:36:57 2022] DELETE /identity/v3/users/251f5164caa84695a4b82a650abee0b5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:58.399608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f9d09-90a0-45af-bd20-3ce27ff48482 service glance] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:58.412199 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f9d09-90a0-45af-bd20-3ce27ff48482 service glance] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:58.475313 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9fff0-b808-47f5-a8f8-7655d1534dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:58.476439 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9fff0-b808-47f5-a8f8-7655d1534dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:58.476941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9fff0-b808-47f5-a8f8-7655d1534dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:58.477222 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9fff0-b808-47f5-a8f8-7655d1534dd7 admin admin] PATH_INFO: `/v3/projects/4c09d709e42448e09e0d8f82ba6372b9` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:58.479335 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9fff0-b808-47f5-a8f8-7655d1534dd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c09d709e42448e09e0d8f82ba6372b9)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:58.481670 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9fff0-b808-47f5-a8f8-7655d1534dd7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:58.541553 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ffd9fff0-b808-47f5-a8f8-7655d1534dd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c09d709e42448e09e0d8f82ba6372b9', 'request_id': 'req-ffd9fff0-b808-47f5-a8f8-7655d1534dd7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:58.548004 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ffd9fff0-b808-47f5-a8f8-7655d1534dd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c09d709e42448e09e0d8f82ba6372b9', 'request_id': 'req-ffd9fff0-b808-47f5-a8f8-7655d1534dd7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:58.554521 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ffd9fff0-b808-47f5-a8f8-7655d1534dd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c09d709e42448e09e0d8f82ba6372b9', 'request_id': 'req-ffd9fff0-b808-47f5-a8f8-7655d1534dd7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:58.568498 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ffd9fff0-b808-47f5-a8f8-7655d1534dd7 admin admin] The token cache is being invalidate because project 4c09d709e42448e09e0d8f82ba6372b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:36:58.568893 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ffd9fff0-b808-47f5-a8f8-7655d1534dd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:58.569559 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ffd9fff0-b808-47f5-a8f8-7655d1534dd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:36:58.571122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1352/2683] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:36:58 2022] DELETE /identity/v3/projects/4c09d709e42448e09e0d8f82ba6372b9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:58.576086 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eec38c-e1b5-4112-a4fb-6a4db7218d38 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:58.576611 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eec38c-e1b5-4112-a4fb-6a4db7218d38 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:58.576771 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eec38c-e1b5-4112-a4fb-6a4db7218d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:58.595973 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-08eec38c-e1b5-4112-a4fb-6a4db7218d38 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:58.597234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-08eec38c-e1b5-4112-a4fb-6a4db7218d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:58.656595 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1332/2684] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:58.665016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f282ef-0fd3-4917-a241-d949758183bf None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:58.665310 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f282ef-0fd3-4917-a241-d949758183bf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:58.665592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f282ef-0fd3-4917-a241-d949758183bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:58.683554 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-30f282ef-0fd3-4917-a241-d949758183bf None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:58.684813 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-30f282ef-0fd3-4917-a241-d949758183bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:58.694475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1353/2685] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:58.697287 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eec38c-e1b5-4112-a4fb-6a4db7218d38 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:58.707291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eec38c-e1b5-4112-a4fb-6a4db7218d38 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:58.713040 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7759fed5-184e-4c8a-8c8c-56474713d90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:58.713728 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759fed5-184e-4c8a-8c8c-56474713d90a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:58.714057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759fed5-184e-4c8a-8c8c-56474713d90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:58.714338 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759fed5-184e-4c8a-8c8c-56474713d90a admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:58.715087 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7759fed5-184e-4c8a-8c8c-56474713d90a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:36:58.715451 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7759fed5-184e-4c8a-8c8c-56474713d90a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:58.717191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7759fed5-184e-4c8a-8c8c-56474713d90a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:58.729840 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1333/2686] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:36:58.734103 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f282ef-0fd3-4917-a241-d949758183bf None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:58.744064 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f282ef-0fd3-4917-a241-d949758183bf None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:58.748789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a4d81-d88b-41b0-9350-cda4293e8731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:58.749439 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a4d81-d88b-41b0-9350-cda4293e8731 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:58.749722 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a4d81-d88b-41b0-9350-cda4293e8731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:58.750002 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a4d81-d88b-41b0-9350-cda4293e8731 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:58.750707 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a4d81-d88b-41b0-9350-cda4293e8731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:58.752672 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a4d81-d88b-41b0-9350-cda4293e8731 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:58.780904 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1354/2687] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:36:58 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:58.785116 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7759fed5-184e-4c8a-8c8c-56474713d90a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:58.795907 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7759fed5-184e-4c8a-8c8c-56474713d90a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:58.852154 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305de204-1131-4ca4-b6aa-98d8adeebf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:58.852865 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305de204-1131-4ca4-b6aa-98d8adeebf60 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:58.853151 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305de204-1131-4ca4-b6aa-98d8adeebf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:58.853432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305de204-1131-4ca4-b6aa-98d8adeebf60 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:58.854160 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305de204-1131-4ca4-b6aa-98d8adeebf60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:58.856076 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305de204-1131-4ca4-b6aa-98d8adeebf60 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:58.878966 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1334/2688] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:36:58 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:36:58.880950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a4d81-d88b-41b0-9350-cda4293e8731 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:58.891636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a4d81-d88b-41b0-9350-cda4293e8731 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:58.897002 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4f820-f77d-4c9e-b84d-a1d00ff3a9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:58.897660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4f820-f77d-4c9e-b84d-a1d00ff3a9bc admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:58.897943 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4f820-f77d-4c9e-b84d-a1d00ff3a9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:58.898219 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4f820-f77d-4c9e-b84d-a1d00ff3a9bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:58.898938 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4f820-f77d-4c9e-b84d-a1d00ff3a9bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:58.900544 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4f820-f77d-4c9e-b84d-a1d00ff3a9bc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:58.911242 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1355/2689] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:58.915412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305de204-1131-4ca4-b6aa-98d8adeebf60 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:58.925374 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305de204-1131-4ca4-b6aa-98d8adeebf60 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:58.930029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43158b-f1d6-4f74-bb75-7cb44bb756b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:58.930732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43158b-f1d6-4f74-bb75-7cb44bb756b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:58.931061 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43158b-f1d6-4f74-bb75-7cb44bb756b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:58.931412 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43158b-f1d6-4f74-bb75-7cb44bb756b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:58.932273 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43158b-f1d6-4f74-bb75-7cb44bb756b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:58.934201 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43158b-f1d6-4f74-bb75-7cb44bb756b4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:58.946066 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1335/2690] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:58.950525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4f820-f77d-4c9e-b84d-a1d00ff3a9bc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:58.961680 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4f820-f77d-4c9e-b84d-a1d00ff3a9bc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:58.966955 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7722065-b10d-4b74-abb1-1eb23234eddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:58.967862 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7722065-b10d-4b74-abb1-1eb23234eddc admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:58.968143 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7722065-b10d-4b74-abb1-1eb23234eddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:58.968415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7722065-b10d-4b74-abb1-1eb23234eddc admin admin] PATH_INFO: `/v3/projects/bbf54128990947488d41f785e3eed445/users/fd32cca92ede4f38a6f417263c2a5182/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:58.970969 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:58.990837 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7722065-b10d-4b74-abb1-1eb23234eddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbf54128990947488d41f785e3eed445, user_id=fd32cca92ede4f38a6f417263c2a5182, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:58.993548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7722065-b10d-4b74-abb1-1eb23234eddc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:59.008159 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1356/2691] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:58 2022] PUT /identity/v3/projects/bbf54128990947488d41f785e3eed445/users/fd32cca92ede4f38a6f417263c2a5182/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:36:59.019300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24882e5-2afd-4d26-b503-09d5efa71229 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:59.019541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24882e5-2afd-4d26-b503-09d5efa71229 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:59.019831 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24882e5-2afd-4d26-b503-09d5efa71229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:59.050105 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a24882e5-2afd-4d26-b503-09d5efa71229 None None] MFA Rules not processed for user `fd32cca92ede4f38a6f417263c2a5182`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:59.051364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a24882e5-2afd-4d26-b503-09d5efa71229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:59.099832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:59.133087 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1336/2692] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:59.137562 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7722065-b10d-4b74-abb1-1eb23234eddc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:59.204822 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7722065-b10d-4b74-abb1-1eb23234eddc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:59.209658 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11889bcd-de07-4644-8100-202e04fa1a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:59.210370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11889bcd-de07-4644-8100-202e04fa1a0f admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:59.210690 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11889bcd-de07-4644-8100-202e04fa1a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:59.210972 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11889bcd-de07-4644-8100-202e04fa1a0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:59.211713 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11889bcd-de07-4644-8100-202e04fa1a0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:59.213659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11889bcd-de07-4644-8100-202e04fa1a0f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:59.242009 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1357/2693] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:36:59 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:59.246193 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24882e5-2afd-4d26-b503-09d5efa71229 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:59.257421 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24882e5-2afd-4d26-b503-09d5efa71229 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:59.316437 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af850dca-4ffd-4724-9f12-97a64ea23a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:59.317170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af850dca-4ffd-4724-9f12-97a64ea23a48 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:59.317460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af850dca-4ffd-4724-9f12-97a64ea23a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:59.318331 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af850dca-4ffd-4724-9f12-97a64ea23a48 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:59.319036 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af850dca-4ffd-4724-9f12-97a64ea23a48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:59.321053 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af850dca-4ffd-4724-9f12-97a64ea23a48 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:59.341220 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1337/2694] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:36:59 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:36:59.345228 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11889bcd-de07-4644-8100-202e04fa1a0f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:59.356890 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11889bcd-de07-4644-8100-202e04fa1a0f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:59.361631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5b602-931d-4caa-9969-3856badee33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:59.362299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5b602-931d-4caa-9969-3856badee33b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:59.362586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5b602-931d-4caa-9969-3856badee33b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:59.362865 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b5b602-931d-4caa-9969-3856badee33b admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:59.363600 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5b602-931d-4caa-9969-3856badee33b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:59.365769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b5b602-931d-4caa-9969-3856badee33b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:59.377001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1358/2695] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:59.381996 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af850dca-4ffd-4724-9f12-97a64ea23a48 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:59.395944 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af850dca-4ffd-4724-9f12-97a64ea23a48 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:59.400741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a3961-7b09-4751-bf18-d30ee1828916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:59.401613 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a3961-7b09-4751-bf18-d30ee1828916 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:59.401899 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a3961-7b09-4751-bf18-d30ee1828916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:59.402177 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a3961-7b09-4751-bf18-d30ee1828916 admin admin] PATH_INFO: `/v3/projects/2c983bfa1bf141b8b598b50b3226a301/users/8af2e575481042dda6e5f0918dfda3eb/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:59.416946 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a3961-7b09-4751-bf18-d30ee1828916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c983bfa1bf141b8b598b50b3226a301, user_id=8af2e575481042dda6e5f0918dfda3eb, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:59.419670 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a3961-7b09-4751-bf18-d30ee1828916 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:59.434937 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1338/2696] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:59 2022] PUT /identity/v3/projects/2c983bfa1bf141b8b598b50b3226a301/users/8af2e575481042dda6e5f0918dfda3eb/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:59.438924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5b602-931d-4caa-9969-3856badee33b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:59.449390 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b5b602-931d-4caa-9969-3856badee33b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:59.508214 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9b507-a385-499f-9de4-8dab12e686d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:59.508927 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9b507-a385-499f-9de4-8dab12e686d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:59.509217 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9b507-a385-499f-9de4-8dab12e686d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:59.509497 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9b507-a385-499f-9de4-8dab12e686d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:59.510226 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9b507-a385-499f-9de4-8dab12e686d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:59.511826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9b507-a385-499f-9de4-8dab12e686d2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:59.521704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1359/2697] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:36:59.526573 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a3961-7b09-4751-bf18-d30ee1828916 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:59.538010 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a3961-7b09-4751-bf18-d30ee1828916 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:59.542703 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3c05e-fe27-4879-9860-2a234217fcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:59.543590 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3c05e-fe27-4879-9860-2a234217fcf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:59.543877 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3c05e-fe27-4879-9860-2a234217fcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:59.544156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3c05e-fe27-4879-9860-2a234217fcf3 admin admin] PATH_INFO: `/v3/projects/2c983bfa1bf141b8b598b50b3226a301/users/8af2e575481042dda6e5f0918dfda3eb/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:59.546223 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e3c05e-fe27-4879-9860-2a234217fcf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c983bfa1bf141b8b598b50b3226a301, user_id=8af2e575481042dda6e5f0918dfda3eb, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:59.549324 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e3c05e-fe27-4879-9860-2a234217fcf3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:59.560697 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1339/2698] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:36:59 2022] PUT /identity/v3/projects/2c983bfa1bf141b8b598b50b3226a301/users/8af2e575481042dda6e5f0918dfda3eb/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:36:59.571161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749792a-5965-4eca-b451-f152421a6b80 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:59.571455 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749792a-5965-4eca-b451-f152421a6b80 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:59.571734 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749792a-5965-4eca-b451-f152421a6b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:59.598127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d749792a-5965-4eca-b451-f152421a6b80 None None] MFA Rules not processed for user `8af2e575481042dda6e5f0918dfda3eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:36:59.599826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d749792a-5965-4eca-b451-f152421a6b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:59.641677 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:36:59.671552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1360/2699] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:36:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:36:59.683019 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3c05e-fe27-4879-9860-2a234217fcf3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:59.684827 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a9e3c05e-fe27-4879-9860-2a234217fcf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:36:59.746584 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3c05e-fe27-4879-9860-2a234217fcf3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:59.754860 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c187cb7-5226-4907-9cf5-29903d3ec8c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:59.755654 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c187cb7-5226-4907-9cf5-29903d3ec8c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:59.756178 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c187cb7-5226-4907-9cf5-29903d3ec8c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:59.756707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c187cb7-5226-4907-9cf5-29903d3ec8c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:59.768664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c187cb7-5226-4907-9cf5-29903d3ec8c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:59.770173 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c187cb7-5226-4907-9cf5-29903d3ec8c5 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:59.780443 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1340/2700] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:36:59.887341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749792a-5965-4eca-b451-f152421a6b80 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:36:59.898115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749792a-5965-4eca-b451-f152421a6b80 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:36:59.901823 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cab4d-a52a-4bb3-b8f9-aff747463167 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:36:59.902486 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cab4d-a52a-4bb3-b8f9-aff747463167 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:36:59.902773 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cab4d-a52a-4bb3-b8f9-aff747463167 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:36:59.903052 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cab4d-a52a-4bb3-b8f9-aff747463167 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:36:59.964577 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3cab4d-a52a-4bb3-b8f9-aff747463167 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:36:59.966038 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3cab4d-a52a-4bb3-b8f9-aff747463167 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:36:59.976519 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1361/2701] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:36:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:02.588120 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c187cb7-5226-4907-9cf5-29903d3ec8c5 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:02.633714 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:02.654886 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c187cb7-5226-4907-9cf5-29903d3ec8c5 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:02.659741 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a901f0-44d7-45ba-b2bf-e01387ebaa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:02.660648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a901f0-44d7-45ba-b2bf-e01387ebaa6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:02.661187 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a901f0-44d7-45ba-b2bf-e01387ebaa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:02.661471 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a901f0-44d7-45ba-b2bf-e01387ebaa6e admin admin] PATH_INFO: `/v3/users/fd32cca92ede4f38a6f417263c2a5182` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:02.662669 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a901f0-44d7-45ba-b2bf-e01387ebaa6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd32cca92ede4f38a6f417263c2a5182)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:02.664670 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a901f0-44d7-45ba-b2bf-e01387ebaa6e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:02.716038 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-20a901f0-44d7-45ba-b2bf-e01387ebaa6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd32cca92ede4f38a6f417263c2a5182'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:02.716734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-20a901f0-44d7-45ba-b2bf-e01387ebaa6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd32cca92ede4f38a6f417263c2a5182'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:02.717344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-20a901f0-44d7-45ba-b2bf-e01387ebaa6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd32cca92ede4f38a6f417263c2a5182'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:02.724698 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-20a901f0-44d7-45ba-b2bf-e01387ebaa6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd32cca92ede4f38a6f417263c2a5182'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:02.738234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-20a901f0-44d7-45ba-b2bf-e01387ebaa6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd32cca92ede4f38a6f417263c2a5182'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:02.760470 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1341/2702] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:37:02 2022] DELETE /identity/v3/users/fd32cca92ede4f38a6f417263c2a5182 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:37:02.765056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf0704-e97b-4716-a89d-d3ea6e417c0e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:02.765537 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf0704-e97b-4716-a89d-d3ea6e417c0e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:02.765752 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf0704-e97b-4716-a89d-d3ea6e417c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:02.785285 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-edaf0704-e97b-4716-a89d-d3ea6e417c0e None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:02.789812 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-edaf0704-e97b-4716-a89d-d3ea6e417c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:02.861515 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1362/2703] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:02.871615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a901f0-44d7-45ba-b2bf-e01387ebaa6e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:02.873069 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-20a901f0-44d7-45ba-b2bf-e01387ebaa6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:02.934218 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a901f0-44d7-45ba-b2bf-e01387ebaa6e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:02.940485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6dbdb-7511-4a33-9bac-a32738c2d5b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:02.941163 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6dbdb-7511-4a33-9bac-a32738c2d5b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:02.941770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6dbdb-7511-4a33-9bac-a32738c2d5b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:02.941770 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6dbdb-7511-4a33-9bac-a32738c2d5b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:02.952953 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6dbdb-7511-4a33-9bac-a32738c2d5b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:02.954475 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6dbdb-7511-4a33-9bac-a32738c2d5b9 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:02.966281 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1342/2704] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:03.452889 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf0704-e97b-4716-a89d-d3ea6e417c0e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:03.454322 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-edaf0704-e97b-4716-a89d-d3ea6e417c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:03.466402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf0704-e97b-4716-a89d-d3ea6e417c0e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:03.471040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe6a52-c93f-4dd0-b419-c724a75475c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:03.471989 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe6a52-c93f-4dd0-b419-c724a75475c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:03.472183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe6a52-c93f-4dd0-b419-c724a75475c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:03.472462 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe6a52-c93f-4dd0-b419-c724a75475c5 admin admin] PATH_INFO: `/v3/projects/bbf54128990947488d41f785e3eed445` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:03.473720 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfe6a52-c93f-4dd0-b419-c724a75475c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbf54128990947488d41f785e3eed445)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:03.475666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfe6a52-c93f-4dd0-b419-c724a75475c5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:03.537005 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dbfe6a52-c93f-4dd0-b419-c724a75475c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbf54128990947488d41f785e3eed445', 'request_id': 'req-dbfe6a52-c93f-4dd0-b419-c724a75475c5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:03.543566 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dbfe6a52-c93f-4dd0-b419-c724a75475c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbf54128990947488d41f785e3eed445', 'request_id': 'req-dbfe6a52-c93f-4dd0-b419-c724a75475c5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:03.550290 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dbfe6a52-c93f-4dd0-b419-c724a75475c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbf54128990947488d41f785e3eed445', 'request_id': 'req-dbfe6a52-c93f-4dd0-b419-c724a75475c5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:03.562254 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:03.570297 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dbfe6a52-c93f-4dd0-b419-c724a75475c5 admin admin] The token cache is being invalidate because project bbf54128990947488d41f785e3eed445 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:37:03.570663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dbfe6a52-c93f-4dd0-b419-c724a75475c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:03.571856 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dbfe6a52-c93f-4dd0-b419-c724a75475c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:03.573809 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1363/2705] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:37:03 2022] DELETE /identity/v3/projects/bbf54128990947488d41f785e3eed445 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:37:03.578024 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6dbdb-7511-4a33-9bac-a32738c2d5b9 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:03.587896 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6dbdb-7511-4a33-9bac-a32738c2d5b9 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:03.656398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442e4b7-4279-4f1c-abb1-609925e7dec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:03.657288 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442e4b7-4279-4f1c-abb1-609925e7dec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:03.657577 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442e4b7-4279-4f1c-abb1-609925e7dec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:03.657885 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442e4b7-4279-4f1c-abb1-609925e7dec6 admin admin] PATH_INFO: `/v3/users/8af2e575481042dda6e5f0918dfda3eb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:03.659111 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442e4b7-4279-4f1c-abb1-609925e7dec6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8af2e575481042dda6e5f0918dfda3eb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:03.661089 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442e4b7-4279-4f1c-abb1-609925e7dec6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:03.664059 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:03.716375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1442e4b7-4279-4f1c-abb1-609925e7dec6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8af2e575481042dda6e5f0918dfda3eb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:03.717065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1442e4b7-4279-4f1c-abb1-609925e7dec6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8af2e575481042dda6e5f0918dfda3eb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:03.717678 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1442e4b7-4279-4f1c-abb1-609925e7dec6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8af2e575481042dda6e5f0918dfda3eb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:03.729545 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1442e4b7-4279-4f1c-abb1-609925e7dec6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8af2e575481042dda6e5f0918dfda3eb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:03.745006 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1442e4b7-4279-4f1c-abb1-609925e7dec6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8af2e575481042dda6e5f0918dfda3eb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:03.759886 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1343/2706] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:37:03 2022] DELETE /identity/v3/users/8af2e575481042dda6e5f0918dfda3eb => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:37:04.337774 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe6a52-c93f-4dd0-b419-c724a75475c5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:04.339965 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dbfe6a52-c93f-4dd0-b419-c724a75475c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:04.355151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe6a52-c93f-4dd0-b419-c724a75475c5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:04.419704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de54aef7-9622-43de-bccf-4a4eff448c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:04.421168 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54aef7-9622-43de-bccf-4a4eff448c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:04.421695 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54aef7-9622-43de-bccf-4a4eff448c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:04.422197 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54aef7-9622-43de-bccf-4a4eff448c9d admin admin] PATH_INFO: `/v3/projects/2c983bfa1bf141b8b598b50b3226a301` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:04.423729 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54aef7-9622-43de-bccf-4a4eff448c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c983bfa1bf141b8b598b50b3226a301)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:04.425743 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54aef7-9622-43de-bccf-4a4eff448c9d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:04.479726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-de54aef7-9622-43de-bccf-4a4eff448c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c983bfa1bf141b8b598b50b3226a301', 'request_id': 'req-de54aef7-9622-43de-bccf-4a4eff448c9d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:04.489143 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-de54aef7-9622-43de-bccf-4a4eff448c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c983bfa1bf141b8b598b50b3226a301', 'request_id': 'req-de54aef7-9622-43de-bccf-4a4eff448c9d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:04.497285 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-de54aef7-9622-43de-bccf-4a4eff448c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c983bfa1bf141b8b598b50b3226a301', 'request_id': 'req-de54aef7-9622-43de-bccf-4a4eff448c9d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:04.519892 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-de54aef7-9622-43de-bccf-4a4eff448c9d admin admin] The token cache is being invalidate because project 2c983bfa1bf141b8b598b50b3226a301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:37:04.520311 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-de54aef7-9622-43de-bccf-4a4eff448c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:04.521651 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-de54aef7-9622-43de-bccf-4a4eff448c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:04.523322 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1364/2707] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:37:04 2022] DELETE /identity/v3/projects/2c983bfa1bf141b8b598b50b3226a301 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:37:04.528327 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9b3fc-a92b-4319-a6c7-93d619e07104 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:04.528676 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9b3fc-a92b-4319-a6c7-93d619e07104 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:04.528957 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9b3fc-a92b-4319-a6c7-93d619e07104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:04.547883 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-54a9b3fc-a92b-4319-a6c7-93d619e07104 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:04.549213 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-54a9b3fc-a92b-4319-a6c7-93d619e07104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:04.614202 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1344/2708] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:04.623080 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558dae3-7acf-42ba-9cc7-2f24cabce209 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:04.623453 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558dae3-7acf-42ba-9cc7-2f24cabce209 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:04.623805 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558dae3-7acf-42ba-9cc7-2f24cabce209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:04.643313 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8558dae3-7acf-42ba-9cc7-2f24cabce209 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:04.644886 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8558dae3-7acf-42ba-9cc7-2f24cabce209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:04.653860 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1365/2709] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:04.658145 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9b3fc-a92b-4319-a6c7-93d619e07104 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:04.668368 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9b3fc-a92b-4319-a6c7-93d619e07104 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:04.673622 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e50355-7243-4200-af03-fa9dbfbf9377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:04.674223 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e50355-7243-4200-af03-fa9dbfbf9377 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:04.674512 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e50355-7243-4200-af03-fa9dbfbf9377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:04.674791 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e50355-7243-4200-af03-fa9dbfbf9377 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:04.675518 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e50355-7243-4200-af03-fa9dbfbf9377 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:37:04.675832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e50355-7243-4200-af03-fa9dbfbf9377 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:04.677515 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e50355-7243-4200-af03-fa9dbfbf9377 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:04.695007 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1345/2710] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:37:04.696144 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558dae3-7acf-42ba-9cc7-2f24cabce209 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:04.705506 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558dae3-7acf-42ba-9cc7-2f24cabce209 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:04.710907 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ebd4a-0719-4a67-82c2-5d49994466ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:04.711567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ebd4a-0719-4a67-82c2-5d49994466ca admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:04.711858 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ebd4a-0719-4a67-82c2-5d49994466ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:04.712163 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ebd4a-0719-4a67-82c2-5d49994466ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:04.712912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ebd4a-0719-4a67-82c2-5d49994466ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:04.714833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ebd4a-0719-4a67-82c2-5d49994466ca admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:04.746366 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1366/2711] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:37:04 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:37:04.750383 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e50355-7243-4200-af03-fa9dbfbf9377 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:04.761656 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e50355-7243-4200-af03-fa9dbfbf9377 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:04.823192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eea255-9d6d-4de0-a80a-007098d38db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:04.823874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eea255-9d6d-4de0-a80a-007098d38db3 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:04.824160 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eea255-9d6d-4de0-a80a-007098d38db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:04.824438 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eea255-9d6d-4de0-a80a-007098d38db3 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:04.825202 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eea255-9d6d-4de0-a80a-007098d38db3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:04.827142 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eea255-9d6d-4de0-a80a-007098d38db3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:04.843030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:04.856720 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1346/2712] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:37:04 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:37:04.858978 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ebd4a-0719-4a67-82c2-5d49994466ca admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:04.870699 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ebd4a-0719-4a67-82c2-5d49994466ca admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:04.876863 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3ba10-b733-42d2-b7d9-36a08d1c6558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:04.877646 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3ba10-b733-42d2-b7d9-36a08d1c6558 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:04.878002 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3ba10-b733-42d2-b7d9-36a08d1c6558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:04.878356 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3ba10-b733-42d2-b7d9-36a08d1c6558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:04.879174 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3ba10-b733-42d2-b7d9-36a08d1c6558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:04.882096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3ba10-b733-42d2-b7d9-36a08d1c6558 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:04.912269 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1367/2713] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:37:04.912646 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26e6db-ae88-41ee-9fe7-ac20dbaacd5d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:04.913057 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26e6db-ae88-41ee-9fe7-ac20dbaacd5d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:04.913341 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26e6db-ae88-41ee-9fe7-ac20dbaacd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:04.916547 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3ba10-b733-42d2-b7d9-36a08d1c6558 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:04.928703 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3ba10-b733-42d2-b7d9-36a08d1c6558 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:04.934550 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c855f-6876-41c5-9f27-bc895992ae50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:04.935245 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9e26e6db-ae88-41ee-9fe7-ac20dbaacd5d None None] MFA Rules not processed for user `cacc328f0f3540299310a0e53afe6acb`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:04.935554 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c855f-6876-41c5-9f27-bc895992ae50 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:04.936077 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c855f-6876-41c5-9f27-bc895992ae50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:04.936475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c855f-6876-41c5-9f27-bc895992ae50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:04.936602 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9e26e6db-ae88-41ee-9fe7-ac20dbaacd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:04.937419 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c855f-6876-41c5-9f27-bc895992ae50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:04.939157 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c855f-6876-41c5-9f27-bc895992ae50 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:04.952131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1368/2714] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:37:04.955059 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c855f-6876-41c5-9f27-bc895992ae50 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:04.965221 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c855f-6876-41c5-9f27-bc895992ae50 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:04.970022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb246d6-5aec-4dae-942e-fab1affdf965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:04.971005 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb246d6-5aec-4dae-942e-fab1affdf965 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:04.971377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb246d6-5aec-4dae-942e-fab1affdf965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:04.971755 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb246d6-5aec-4dae-942e-fab1affdf965 admin admin] PATH_INFO: `/v3/projects/9d9c73a75a0f431892c67d62a509422d/users/2d2a269b11654ed5b2437e089a5a3be5/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:04.988054 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb246d6-5aec-4dae-942e-fab1affdf965 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d9c73a75a0f431892c67d62a509422d, user_id=2d2a269b11654ed5b2437e089a5a3be5, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:04.991217 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb246d6-5aec-4dae-942e-fab1affdf965 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:04.999524 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1347/2715] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:37:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:05.009847 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1369/2716] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:37:04 2022] PUT /identity/v3/projects/9d9c73a75a0f431892c67d62a509422d/users/2d2a269b11654ed5b2437e089a5a3be5/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:37:05.020021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4144480-72ce-4bbe-a039-06736f069e5b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:05.020319 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4144480-72ce-4bbe-a039-06736f069e5b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:05.020673 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4144480-72ce-4bbe-a039-06736f069e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:05.050189 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c4144480-72ce-4bbe-a039-06736f069e5b None None] MFA Rules not processed for user `2d2a269b11654ed5b2437e089a5a3be5`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:05.051482 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c4144480-72ce-4bbe-a039-06736f069e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:05.126155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1348/2717] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:37:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:05.139152 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb246d6-5aec-4dae-942e-fab1affdf965 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:05.141233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4cb246d6-5aec-4dae-942e-fab1affdf965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:05.162848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:05.208839 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb246d6-5aec-4dae-942e-fab1affdf965 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:05.212459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597180d-3393-461d-8f0f-58e115cad676 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cee4d23797e430e9eeddc394ed7f50e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:05.213131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597180d-3393-461d-8f0f-58e115cad676 service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:05.213414 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597180d-3393-461d-8f0f-58e115cad676 service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:05.213691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597180d-3393-461d-8f0f-58e115cad676 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:05.222022 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597180d-3393-461d-8f0f-58e115cad676 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:05.223407 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597180d-3393-461d-8f0f-58e115cad676 service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:05.233584 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1370/2718] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:06.123343 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4144480-72ce-4bbe-a039-06736f069e5b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:06.124801 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c4144480-72ce-4bbe-a039-06736f069e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:06.222119 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4144480-72ce-4bbe-a039-06736f069e5b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:06.227982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d34a9f-7729-4f81-a01c-dd67118cab6a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd07165f554a445169b2c3f0d173c0bb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:06.228714 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d34a9f-7729-4f81-a01c-dd67118cab6a service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:06.229009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d34a9f-7729-4f81-a01c-dd67118cab6a service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:06.229293 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d34a9f-7729-4f81-a01c-dd67118cab6a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:06.231246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d1d34a9f-7729-4f81-a01c-dd67118cab6a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:06.244058 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d34a9f-7729-4f81-a01c-dd67118cab6a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:06.246720 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d34a9f-7729-4f81-a01c-dd67118cab6a service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:06.263655 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1349/2719] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:15.229542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=19 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:16.258982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:26.945303 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597180d-3393-461d-8f0f-58e115cad676 service glance] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:26.947206 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0597180d-3393-461d-8f0f-58e115cad676 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:27.011167 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597180d-3393-461d-8f0f-58e115cad676 service glance] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:27.015613 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb6445-369e-4ee5-a177-a80838639778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:27.016477 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb6445-369e-4ee5-a177-a80838639778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:27.016791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb6445-369e-4ee5-a177-a80838639778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:27.017067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb6445-369e-4ee5-a177-a80838639778 admin admin] PATH_INFO: `/v3/users/199f0ee6813f484ca3a390c2bc4600fd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:27.018243 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bb6445-369e-4ee5-a177-a80838639778 admin admin] RBAC: Authorizing `identity:delete_user(user_id=199f0ee6813f484ca3a390c2bc4600fd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:27.020170 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bb6445-369e-4ee5-a177-a80838639778 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:27.051208 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d34a9f-7729-4f81-a01c-dd67118cab6a service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:27.061653 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d34a9f-7729-4f81-a01c-dd67118cab6a service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:27.066127 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f38ff-f525-4c64-bfc7-dededbdd040f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:27.066885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d7bb6445-369e-4ee5-a177-a80838639778 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '199f0ee6813f484ca3a390c2bc4600fd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.067018 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f38ff-f525-4c64-bfc7-dededbdd040f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:27.067250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f38ff-f525-4c64-bfc7-dededbdd040f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:27.067522 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f38ff-f525-4c64-bfc7-dededbdd040f admin admin] PATH_INFO: `/v3/users/2d2a269b11654ed5b2437e089a5a3be5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:27.068758 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f38ff-f525-4c64-bfc7-dededbdd040f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d2a269b11654ed5b2437e089a5a3be5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:27.070694 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f38ff-f525-4c64-bfc7-dededbdd040f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:27.079267 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d7bb6445-369e-4ee5-a177-a80838639778 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '199f0ee6813f484ca3a390c2bc4600fd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.079980 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d7bb6445-369e-4ee5-a177-a80838639778 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '199f0ee6813f484ca3a390c2bc4600fd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.080653 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d7bb6445-369e-4ee5-a177-a80838639778 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '199f0ee6813f484ca3a390c2bc4600fd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.089744 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d7bb6445-369e-4ee5-a177-a80838639778 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '199f0ee6813f484ca3a390c2bc4600fd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.104413 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1371/2720] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:37:26 2022] DELETE /identity/v3/users/199f0ee6813f484ca3a390c2bc4600fd => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:37:27.109165 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14eb39-3096-477e-91b8-b206395661a0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:27.109456 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14eb39-3096-477e-91b8-b206395661a0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:27.109756 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14eb39-3096-477e-91b8-b206395661a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:27.120182 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cd9f38ff-f525-4c64-bfc7-dededbdd040f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d2a269b11654ed5b2437e089a5a3be5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.120886 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cd9f38ff-f525-4c64-bfc7-dededbdd040f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d2a269b11654ed5b2437e089a5a3be5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.121839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cd9f38ff-f525-4c64-bfc7-dededbdd040f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d2a269b11654ed5b2437e089a5a3be5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.127772 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9d14eb39-3096-477e-91b8-b206395661a0 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:27.129010 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9d14eb39-3096-477e-91b8-b206395661a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:27.129437 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cd9f38ff-f525-4c64-bfc7-dededbdd040f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d2a269b11654ed5b2437e089a5a3be5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.142558 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cd9f38ff-f525-4c64-bfc7-dededbdd040f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d2a269b11654ed5b2437e089a5a3be5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.157357 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1350/2721] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:37:27 2022] DELETE /identity/v3/users/2d2a269b11654ed5b2437e089a5a3be5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:37:27.161650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f6030-5eb9-4281-bd27-ede88f35ff03 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:27.161945 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f6030-5eb9-4281-bd27-ede88f35ff03 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:27.162225 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f6030-5eb9-4281-bd27-ede88f35ff03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:27.180974 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c72f6030-5eb9-4281-bd27-ede88f35ff03 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:27.182628 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c72f6030-5eb9-4281-bd27-ede88f35ff03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:27.198972 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1372/2722] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:27.208525 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14eb39-3096-477e-91b8-b206395661a0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:27.209921 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9d14eb39-3096-477e-91b8-b206395661a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:27.233320 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:27.245227 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1351/2723] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:37:27.254686 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f6030-5eb9-4281-bd27-ede88f35ff03 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:27.256021 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c72f6030-5eb9-4281-bd27-ede88f35ff03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:27.292129 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14eb39-3096-477e-91b8-b206395661a0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:27.312125 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f6030-5eb9-4281-bd27-ede88f35ff03 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:27.315710 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8c417-0ca6-4b31-9dc6-e12a43094938 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:27.316346 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8c417-0ca6-4b31-9dc6-e12a43094938 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:27.316677 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8c417-0ca6-4b31-9dc6-e12a43094938 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:27.316954 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8c417-0ca6-4b31-9dc6-e12a43094938 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:27.327522 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8c417-0ca6-4b31-9dc6-e12a43094938 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:27.328928 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8c417-0ca6-4b31-9dc6-e12a43094938 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:27.338599 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1352/2724] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:37:27.345780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448827f-a25e-426f-9bca-ee4979aa08fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:27.346428 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448827f-a25e-426f-9bca-ee4979aa08fe service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:27.346712 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448827f-a25e-426f-9bca-ee4979aa08fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:27.346988 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448827f-a25e-426f-9bca-ee4979aa08fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:27.356984 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448827f-a25e-426f-9bca-ee4979aa08fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:27.358434 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448827f-a25e-426f-9bca-ee4979aa08fe service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:27.369306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1373/2725] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:37:27.744237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8c417-0ca6-4b31-9dc6-e12a43094938 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:27.745729 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b8e8c417-0ca6-4b31-9dc6-e12a43094938 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:27.758170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8c417-0ca6-4b31-9dc6-e12a43094938 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:27.763036 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2401bb-4481-4aeb-ba92-7f96e63302f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:27.773469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2401bb-4481-4aeb-ba92-7f96e63302f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:27.773755 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2401bb-4481-4aeb-ba92-7f96e63302f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:27.774035 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2401bb-4481-4aeb-ba92-7f96e63302f2 admin admin] PATH_INFO: `/v3/projects/9d9c73a75a0f431892c67d62a509422d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:27.775577 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2401bb-4481-4aeb-ba92-7f96e63302f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d9c73a75a0f431892c67d62a509422d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:27.777541 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2401bb-4481-4aeb-ba92-7f96e63302f2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:27.789084 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:27.799605 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448827f-a25e-426f-9bca-ee4979aa08fe service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:27.801412 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6448827f-a25e-426f-9bca-ee4979aa08fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:27.818342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448827f-a25e-426f-9bca-ee4979aa08fe service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:27.824097 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc9892-1d56-4b2c-9464-358dcf340aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:27.825091 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc9892-1d56-4b2c-9464-358dcf340aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:27.825441 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc9892-1d56-4b2c-9464-358dcf340aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:27.825789 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc9892-1d56-4b2c-9464-358dcf340aaf admin admin] PATH_INFO: `/v3/projects/79d1632d8aa441b782effcc0008b9631` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:27.826998 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cc9892-1d56-4b2c-9464-358dcf340aaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=79d1632d8aa441b782effcc0008b9631)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:27.828975 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cc9892-1d56-4b2c-9464-358dcf340aaf admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:27.837554 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d2401bb-4481-4aeb-ba92-7f96e63302f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d9c73a75a0f431892c67d62a509422d', 'request_id': 'req-3d2401bb-4481-4aeb-ba92-7f96e63302f2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.843802 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d2401bb-4481-4aeb-ba92-7f96e63302f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d9c73a75a0f431892c67d62a509422d', 'request_id': 'req-3d2401bb-4481-4aeb-ba92-7f96e63302f2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.850718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d2401bb-4481-4aeb-ba92-7f96e63302f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d9c73a75a0f431892c67d62a509422d', 'request_id': 'req-3d2401bb-4481-4aeb-ba92-7f96e63302f2'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.864763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d2401bb-4481-4aeb-ba92-7f96e63302f2 admin admin] The token cache is being invalidate because project 9d9c73a75a0f431892c67d62a509422d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:37:27.865146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d2401bb-4481-4aeb-ba92-7f96e63302f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.865839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d2401bb-4481-4aeb-ba92-7f96e63302f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.867398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1353/2726] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:37:27 2022] DELETE /identity/v3/projects/9d9c73a75a0f431892c67d62a509422d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:37:27.872674 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61817321-7828-4dc1-bcea-a602ab53d3ce None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:27.872970 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61817321-7828-4dc1-bcea-a602ab53d3ce None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:27.873257 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61817321-7828-4dc1-bcea-a602ab53d3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:27.879755 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-91cc9892-1d56-4b2c-9464-358dcf340aaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79d1632d8aa441b782effcc0008b9631', 'request_id': 'req-91cc9892-1d56-4b2c-9464-358dcf340aaf'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.886273 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-91cc9892-1d56-4b2c-9464-358dcf340aaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79d1632d8aa441b782effcc0008b9631', 'request_id': 'req-91cc9892-1d56-4b2c-9464-358dcf340aaf'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.890904 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-61817321-7828-4dc1-bcea-a602ab53d3ce None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:27.892119 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-61817321-7828-4dc1-bcea-a602ab53d3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:27.893126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-91cc9892-1d56-4b2c-9464-358dcf340aaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79d1632d8aa441b782effcc0008b9631', 'request_id': 'req-91cc9892-1d56-4b2c-9464-358dcf340aaf'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.906916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-91cc9892-1d56-4b2c-9464-358dcf340aaf admin admin] The token cache is being invalidate because project 79d1632d8aa441b782effcc0008b9631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:37:27.907204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-91cc9892-1d56-4b2c-9464-358dcf340aaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.907869 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-91cc9892-1d56-4b2c-9464-358dcf340aaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:27.912824 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1374/2727] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:37:27 2022] DELETE /identity/v3/projects/79d1632d8aa441b782effcc0008b9631 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:37:27.914502 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8041a4-9d31-4063-8039-695704fcf793 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:27.914645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8041a4-9d31-4063-8039-695704fcf793 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:27.915506 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8041a4-9d31-4063-8039-695704fcf793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:27.935292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7d8041a4-9d31-4063-8039-695704fcf793 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:27.936615 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7d8041a4-9d31-4063-8039-695704fcf793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:27.960660 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1354/2728] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:37:27.970104 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e579fde-7216-486b-a283-8a801e58fa9f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:27.970432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e579fde-7216-486b-a283-8a801e58fa9f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:27.970668 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e579fde-7216-486b-a283-8a801e58fa9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:27.983079 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:27.994285 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6e579fde-7216-486b-a283-8a801e58fa9f None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:27.995498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6e579fde-7216-486b-a283-8a801e58fa9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:28.004571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1355/2729] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:28.008805 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1375/2730] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:37:28.012790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e579fde-7216-486b-a283-8a801e58fa9f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.016894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c544116-21f1-49f3-8278-7055a94247c5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.017196 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c544116-21f1-49f3-8278-7055a94247c5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.017541 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c544116-21f1-49f3-8278-7055a94247c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.022721 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e579fde-7216-486b-a283-8a801e58fa9f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.029508 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6625cc6-d7ed-4860-8799-9189ed8867db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.030171 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6625cc6-d7ed-4860-8799-9189ed8867db admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.030459 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6625cc6-d7ed-4860-8799-9189ed8867db admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.032791 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6625cc6-d7ed-4860-8799-9189ed8867db admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.032791 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6625cc6-d7ed-4860-8799-9189ed8867db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:37:28.032791 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6625cc6-d7ed-4860-8799-9189ed8867db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.042677 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6625cc6-d7ed-4860-8799-9189ed8867db admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.056448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6c544116-21f1-49f3-8278-7055a94247c5 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:28.056680 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1356/2731] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:37:28.057566 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6c544116-21f1-49f3-8278-7055a94247c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:28.060539 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6625cc6-d7ed-4860-8799-9189ed8867db admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.066528 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1376/2732] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:37:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:37:28.070392 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c544116-21f1-49f3-8278-7055a94247c5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.073366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6625cc6-d7ed-4860-8799-9189ed8867db admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.078237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9ad3e-77cb-4710-9731-683ca8a65609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.078901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9ad3e-77cb-4710-9731-683ca8a65609 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.079189 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9ad3e-77cb-4710-9731-683ca8a65609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.079468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9ad3e-77cb-4710-9731-683ca8a65609 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.079597 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c544116-21f1-49f3-8278-7055a94247c5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.080253 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9ad3e-77cb-4710-9731-683ca8a65609 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.082220 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9ad3e-77cb-4710-9731-683ca8a65609 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.085176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34348540-fad7-4f98-8682-b71c1d4528cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.085767 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34348540-fad7-4f98-8682-b71c1d4528cd admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.086051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34348540-fad7-4f98-8682-b71c1d4528cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.086326 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34348540-fad7-4f98-8682-b71c1d4528cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.087605 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34348540-fad7-4f98-8682-b71c1d4528cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:37:28.087912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34348540-fad7-4f98-8682-b71c1d4528cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.089545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34348540-fad7-4f98-8682-b71c1d4528cd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.111205 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1377/2733] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 10:37:28.112433 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34348540-fad7-4f98-8682-b71c1d4528cd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.122588 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1357/2734] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:37:28 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:37:28.127300 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9ad3e-77cb-4710-9731-683ca8a65609 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.130183 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34348540-fad7-4f98-8682-b71c1d4528cd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.197960 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:28.203575 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9ad3e-77cb-4710-9731-683ca8a65609 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.207848 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd1d71-d71f-4ec4-a0b8-795787eb0c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.208543 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd1d71-d71f-4ec4-a0b8-795787eb0c17 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.208850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd1d71-d71f-4ec4-a0b8-795787eb0c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.209122 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd1d71-d71f-4ec4-a0b8-795787eb0c17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.209906 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd1d71-d71f-4ec4-a0b8-795787eb0c17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.211198 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2d9bc-0e10-45a5-9feb-7bf48434cef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.212553 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd1d71-d71f-4ec4-a0b8-795787eb0c17 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.213161 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2d9bc-0e10-45a5-9feb-7bf48434cef3 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.213750 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2d9bc-0e10-45a5-9feb-7bf48434cef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.214071 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2d9bc-0e10-45a5-9feb-7bf48434cef3 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.214915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2d9bc-0e10-45a5-9feb-7bf48434cef3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.217861 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2d9bc-0e10-45a5-9feb-7bf48434cef3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.245487 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1358/2735] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:37:28 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:37:28.247098 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2d9bc-0e10-45a5-9feb-7bf48434cef3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.247935 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1378/2736] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:37:28 2022] POST /identity/v3/projects => generated 363 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:37:28.251832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd1d71-d71f-4ec4-a0b8-795787eb0c17 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.258896 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2d9bc-0e10-45a5-9feb-7bf48434cef3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.315206 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd1d71-d71f-4ec4-a0b8-795787eb0c17 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.317957 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8e304-e6a7-43d0-bbe5-27a846f4004a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.318633 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8e304-e6a7-43d0-bbe5-27a846f4004a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.318922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8e304-e6a7-43d0-bbe5-27a846f4004a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.319202 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8e304-e6a7-43d0-bbe5-27a846f4004a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.319934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8e304-e6a7-43d0-bbe5-27a846f4004a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.320224 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a238c6f-d991-412d-a40b-9383f308094c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.321495 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a238c6f-d991-412d-a40b-9383f308094c admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.321792 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a238c6f-d991-412d-a40b-9383f308094c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.322074 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a238c6f-d991-412d-a40b-9383f308094c admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.322224 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8e304-e6a7-43d0-bbe5-27a846f4004a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.322769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a238c6f-d991-412d-a40b-9383f308094c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.324776 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a238c6f-d991-412d-a40b-9383f308094c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.333223 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1359/2737] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:37:28.338019 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8e304-e6a7-43d0-bbe5-27a846f4004a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.348982 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8e304-e6a7-43d0-bbe5-27a846f4004a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.353047 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1379/2738] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:37:28 2022] POST /identity/v3/users => generated 282 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:37:28.353877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a238c6f-d991-412d-a40b-9383f308094c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.354627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5540e5-9c84-44ec-93b1-61cf61a162ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.355318 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5540e5-9c84-44ec-93b1-61cf61a162ef admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.355607 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5540e5-9c84-44ec-93b1-61cf61a162ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.355892 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5540e5-9c84-44ec-93b1-61cf61a162ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.356654 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5540e5-9c84-44ec-93b1-61cf61a162ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.358255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5540e5-9c84-44ec-93b1-61cf61a162ef admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.363534 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a238c6f-d991-412d-a40b-9383f308094c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.368788 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61d5f0-6539-4639-8566-fa32a0dbcdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.369180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1360/2739] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:37:28.369439 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61d5f0-6539-4639-8566-fa32a0dbcdd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.369726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61d5f0-6539-4639-8566-fa32a0dbcdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.370007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61d5f0-6539-4639-8566-fa32a0dbcdd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.370732 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61d5f0-6539-4639-8566-fa32a0dbcdd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.372400 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61d5f0-6539-4639-8566-fa32a0dbcdd7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.373158 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5540e5-9c84-44ec-93b1-61cf61a162ef admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.385970 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1380/2740] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:37:28.386148 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5540e5-9c84-44ec-93b1-61cf61a162ef admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.387301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61d5f0-6539-4639-8566-fa32a0dbcdd7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.388267 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab4df8-cb4a-4db8-9c50-e2f5fdf04b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.389176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab4df8-cb4a-4db8-9c50-e2f5fdf04b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.389466 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab4df8-cb4a-4db8-9c50-e2f5fdf04b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.389746 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab4df8-cb4a-4db8-9c50-e2f5fdf04b26 admin admin] PATH_INFO: `/v3/projects/67ed50546d004a459ad43df30992b3d2/users/bf3bcd9f595d499eab20a4180b3d8461/roles/2820090979cb487db491f835e03e6471` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.397131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61d5f0-6539-4639-8566-fa32a0dbcdd7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.402607 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8608092-8669-42c1-ac6b-72ebb1cc7ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.403250 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8608092-8669-42c1-ac6b-72ebb1cc7ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.403539 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8608092-8669-42c1-ac6b-72ebb1cc7ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.403829 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8608092-8669-42c1-ac6b-72ebb1cc7ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.404573 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8608092-8669-42c1-ac6b-72ebb1cc7ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.405558 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab4df8-cb4a-4db8-9c50-e2f5fdf04b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ed50546d004a459ad43df30992b3d2, user_id=bf3bcd9f595d499eab20a4180b3d8461, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.406163 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8608092-8669-42c1-ac6b-72ebb1cc7ad8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.408345 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab4df8-cb4a-4db8-9c50-e2f5fdf04b26 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.418072 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1381/2741] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:37:28.422230 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8608092-8669-42c1-ac6b-72ebb1cc7ad8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.425032 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1361/2742] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:37:28 2022] PUT /identity/v3/projects/67ed50546d004a459ad43df30992b3d2/users/bf3bcd9f595d499eab20a4180b3d8461/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:37:28.437161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8608092-8669-42c1-ac6b-72ebb1cc7ad8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.451611 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:28.492966 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c08e7-c1bc-424e-b57b-0f44709f6f5c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.493270 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c08e7-c1bc-424e-b57b-0f44709f6f5c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.493785 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c08e7-c1bc-424e-b57b-0f44709f6f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.503454 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48046795-f90b-4132-96ff-8e5adc5175e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.504308 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48046795-f90b-4132-96ff-8e5adc5175e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.504618 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48046795-f90b-4132-96ff-8e5adc5175e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.504898 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48046795-f90b-4132-96ff-8e5adc5175e0 admin admin] PATH_INFO: `/v3/projects/43922a0d85f44e378821e7bd9e956e63/users/794b6ddd679443f584e0903f0216fb64/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.520238 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-401c08e7-c1bc-424e-b57b-0f44709f6f5c None None] MFA Rules not processed for user `bf3bcd9f595d499eab20a4180b3d8461`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:28.520439 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48046795-f90b-4132-96ff-8e5adc5175e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43922a0d85f44e378821e7bd9e956e63, user_id=794b6ddd679443f584e0903f0216fb64, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.521579 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-401c08e7-c1bc-424e-b57b-0f44709f6f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:28.523136 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48046795-f90b-4132-96ff-8e5adc5175e0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.536962 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1382/2743] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:37:28 2022] PUT /identity/v3/projects/43922a0d85f44e378821e7bd9e956e63/users/794b6ddd679443f584e0903f0216fb64/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:37:28.547507 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3479e1-3a27-4ea3-9249-fe218e020706 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.547809 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3479e1-3a27-4ea3-9249-fe218e020706 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.548092 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3479e1-3a27-4ea3-9249-fe218e020706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.574990 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-cf3479e1-3a27-4ea3-9249-fe218e020706 None None] MFA Rules not processed for user `794b6ddd679443f584e0903f0216fb64`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:28.576204 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cf3479e1-3a27-4ea3-9249-fe218e020706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:28.591000 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1362/2744] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:37:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:37:28.595502 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c08e7-c1bc-424e-b57b-0f44709f6f5c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.606070 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c08e7-c1bc-424e-b57b-0f44709f6f5c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.642144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:28.647315 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1383/2745] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:37:28 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:37:28.655238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3479e1-3a27-4ea3-9249-fe218e020706 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.656676 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cf3479e1-3a27-4ea3-9249-fe218e020706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:28.668192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afbfa9-7d8e-4cd0-a70b-abee4228d8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.668924 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afbfa9-7d8e-4cd0-a70b-abee4228d8b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.669213 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afbfa9-7d8e-4cd0-a70b-abee4228d8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.669498 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afbfa9-7d8e-4cd0-a70b-abee4228d8b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.670192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82afbfa9-7d8e-4cd0-a70b-abee4228d8b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.672134 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82afbfa9-7d8e-4cd0-a70b-abee4228d8b4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.698210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1363/2746] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:37:28 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:37:28.702214 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afbfa9-7d8e-4cd0-a70b-abee4228d8b4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.742527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3479e1-3a27-4ea3-9249-fe218e020706 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.763340 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:28.763863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afbfa9-7d8e-4cd0-a70b-abee4228d8b4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.769009 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cef68e-9af2-4fb7-9e41-3f3434b5f222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.769606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cef68e-9af2-4fb7-9e41-3f3434b5f222 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.769893 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cef68e-9af2-4fb7-9e41-3f3434b5f222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.770175 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cef68e-9af2-4fb7-9e41-3f3434b5f222 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.770863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cef68e-9af2-4fb7-9e41-3f3434b5f222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.773167 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cef68e-9af2-4fb7-9e41-3f3434b5f222 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.795882 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b52640-3f48-4d76-b775-d9a46f5b7536 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.796568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b52640-3f48-4d76-b775-d9a46f5b7536 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.796859 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b52640-3f48-4d76-b775-d9a46f5b7536 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.797142 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b52640-3f48-4d76-b775-d9a46f5b7536 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.801268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1364/2747] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:37:28 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:37:28.802365 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cef68e-9af2-4fb7-9e41-3f3434b5f222 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.812567 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cef68e-9af2-4fb7-9e41-3f3434b5f222 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.817194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b66add-8d43-484d-af27-9914d8529b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.817872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b66add-8d43-484d-af27-9914d8529b65 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.818160 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b66add-8d43-484d-af27-9914d8529b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.818654 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b66add-8d43-484d-af27-9914d8529b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.819163 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b66add-8d43-484d-af27-9914d8529b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.820829 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b66add-8d43-484d-af27-9914d8529b65 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.834637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1365/2748] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:37:28.835632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b66add-8d43-484d-af27-9914d8529b65 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.845325 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b66add-8d43-484d-af27-9914d8529b65 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.850582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c3696-cb71-408d-abd2-c513945569a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.851748 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c3696-cb71-408d-abd2-c513945569a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.852038 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c3696-cb71-408d-abd2-c513945569a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.852348 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c3696-cb71-408d-abd2-c513945569a2 admin admin] PATH_INFO: `/v3/projects/956c49b590f24ccb8b4b83a5177ba172/users/acd0c9a0cce94452a52940cdba26937a/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.856009 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b52640-3f48-4d76-b775-d9a46f5b7536 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.857485 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b52640-3f48-4d76-b775-d9a46f5b7536 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.866833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1384/2749] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:28 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:37:28.867248 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c3696-cb71-408d-abd2-c513945569a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=956c49b590f24ccb8b4b83a5177ba172, user_id=acd0c9a0cce94452a52940cdba26937a, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.870482 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c3696-cb71-408d-abd2-c513945569a2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.884004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1366/2750] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:37:28 2022] PUT /identity/v3/projects/956c49b590f24ccb8b4b83a5177ba172/users/acd0c9a0cce94452a52940cdba26937a/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:37:28.888148 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b52640-3f48-4d76-b775-d9a46f5b7536 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.898857 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b52640-3f48-4d76-b775-d9a46f5b7536 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:28.964697 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba381e-0cda-4b48-9ed5-04045273ec43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:28.965526 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba381e-0cda-4b48-9ed5-04045273ec43 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:28.965939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba381e-0cda-4b48-9ed5-04045273ec43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:28.966224 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba381e-0cda-4b48-9ed5-04045273ec43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:28.966966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba381e-0cda-4b48-9ed5-04045273ec43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:28.968760 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba381e-0cda-4b48-9ed5-04045273ec43 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:28.983109 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1385/2751] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:37:28.984326 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c3696-cb71-408d-abd2-c513945569a2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:28.994715 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c3696-cb71-408d-abd2-c513945569a2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:29.000148 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7024825-26d0-495f-884e-e0a3b5a717fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:29.001070 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7024825-26d0-495f-884e-e0a3b5a717fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:29.001356 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7024825-26d0-495f-884e-e0a3b5a717fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:29.001635 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7024825-26d0-495f-884e-e0a3b5a717fe admin admin] PATH_INFO: `/v3/projects/956c49b590f24ccb8b4b83a5177ba172/users/acd0c9a0cce94452a52940cdba26937a/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:29.003615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7024825-26d0-495f-884e-e0a3b5a717fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=956c49b590f24ccb8b4b83a5177ba172, user_id=acd0c9a0cce94452a52940cdba26937a, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:29.006361 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7024825-26d0-495f-884e-e0a3b5a717fe admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:29.018117 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1367/2752] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:37:28 2022] PUT /identity/v3/projects/956c49b590f24ccb8b4b83a5177ba172/users/acd0c9a0cce94452a52940cdba26937a/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:37:29.030656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f33b6-6f10-4ce5-8b9b-44ca96679307 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:29.030978 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f33b6-6f10-4ce5-8b9b-44ca96679307 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:29.031646 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f33b6-6f10-4ce5-8b9b-44ca96679307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:29.060685 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-289f33b6-6f10-4ce5-8b9b-44ca96679307 None None] MFA Rules not processed for user `acd0c9a0cce94452a52940cdba26937a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:29.061923 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-289f33b6-6f10-4ce5-8b9b-44ca96679307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:29.132908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:29.137941 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1386/2753] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:37:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:29.152012 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7024825-26d0-495f-884e-e0a3b5a717fe admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:29.228640 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7024825-26d0-495f-884e-e0a3b5a717fe admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:29.232252 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b6c0e-372c-4b11-bd5e-b474087a32f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:29.233618 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b6c0e-372c-4b11-bd5e-b474087a32f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:29.233999 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b6c0e-372c-4b11-bd5e-b474087a32f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:29.234416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b6c0e-372c-4b11-bd5e-b474087a32f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:29.251828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:29.294534 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b6c0e-372c-4b11-bd5e-b474087a32f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:29.296011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b6c0e-372c-4b11-bd5e-b474087a32f5 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:29.306344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1368/2754] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:31.840795 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f33b6-6f10-4ce5-8b9b-44ca96679307 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:31.909591 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f33b6-6f10-4ce5-8b9b-44ca96679307 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:31.915586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ea85c-0144-4e2d-bf18-94fdd84eae95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:31.916460 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ea85c-0144-4e2d-bf18-94fdd84eae95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:31.916777 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ea85c-0144-4e2d-bf18-94fdd84eae95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:31.917055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ea85c-0144-4e2d-bf18-94fdd84eae95 admin admin] PATH_INFO: `/v3/users/bf3bcd9f595d499eab20a4180b3d8461` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:31.918262 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ea85c-0144-4e2d-bf18-94fdd84eae95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf3bcd9f595d499eab20a4180b3d8461)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:31.920195 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ea85c-0144-4e2d-bf18-94fdd84eae95 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:31.966976 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5c7ea85c-0144-4e2d-bf18-94fdd84eae95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf3bcd9f595d499eab20a4180b3d8461'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:31.979446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5c7ea85c-0144-4e2d-bf18-94fdd84eae95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf3bcd9f595d499eab20a4180b3d8461'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:31.980101 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5c7ea85c-0144-4e2d-bf18-94fdd84eae95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf3bcd9f595d499eab20a4180b3d8461'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:31.980725 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5c7ea85c-0144-4e2d-bf18-94fdd84eae95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf3bcd9f595d499eab20a4180b3d8461'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:31.988210 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5c7ea85c-0144-4e2d-bf18-94fdd84eae95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf3bcd9f595d499eab20a4180b3d8461'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:32.000496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1387/2755] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:37:31 2022] DELETE /identity/v3/users/bf3bcd9f595d499eab20a4180b3d8461 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:37:32.005344 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71016490-0e1c-4a75-a643-cd50d4c147ec None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:32.005846 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71016490-0e1c-4a75-a643-cd50d4c147ec None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:32.005963 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71016490-0e1c-4a75-a643-cd50d4c147ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:32.023764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-71016490-0e1c-4a75-a643-cd50d4c147ec None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:32.025789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-71016490-0e1c-4a75-a643-cd50d4c147ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:32.084979 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1369/2756] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:32.096808 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ea85c-0144-4e2d-bf18-94fdd84eae95 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:32.098296 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5c7ea85c-0144-4e2d-bf18-94fdd84eae95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:32.166839 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ea85c-0144-4e2d-bf18-94fdd84eae95 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:32.170452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04d1ec-d9bd-4a73-b4dc-d6ac34e78e4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:32.171094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04d1ec-d9bd-4a73-b4dc-d6ac34e78e4c service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:32.171377 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04d1ec-d9bd-4a73-b4dc-d6ac34e78e4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:32.171652 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04d1ec-d9bd-4a73-b4dc-d6ac34e78e4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:32.181202 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da04d1ec-d9bd-4a73-b4dc-d6ac34e78e4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:32.182675 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da04d1ec-d9bd-4a73-b4dc-d6ac34e78e4c service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:32.192461 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1388/2757] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:32.453249 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71016490-0e1c-4a75-a643-cd50d4c147ec None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:32.454805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-71016490-0e1c-4a75-a643-cd50d4c147ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:32.467352 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71016490-0e1c-4a75-a643-cd50d4c147ec None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:32.472446 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16c2a6-08b7-457b-a01a-203cadec3c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:32.473433 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16c2a6-08b7-457b-a01a-203cadec3c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:32.473876 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16c2a6-08b7-457b-a01a-203cadec3c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:32.474155 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16c2a6-08b7-457b-a01a-203cadec3c41 admin admin] PATH_INFO: `/v3/projects/67ed50546d004a459ad43df30992b3d2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:32.475598 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16c2a6-08b7-457b-a01a-203cadec3c41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67ed50546d004a459ad43df30992b3d2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:32.478266 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16c2a6-08b7-457b-a01a-203cadec3c41 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:32.525126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb16c2a6-08b7-457b-a01a-203cadec3c41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67ed50546d004a459ad43df30992b3d2', 'request_id': 'req-cb16c2a6-08b7-457b-a01a-203cadec3c41'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:32.531160 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb16c2a6-08b7-457b-a01a-203cadec3c41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67ed50546d004a459ad43df30992b3d2', 'request_id': 'req-cb16c2a6-08b7-457b-a01a-203cadec3c41'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:32.537296 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb16c2a6-08b7-457b-a01a-203cadec3c41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67ed50546d004a459ad43df30992b3d2', 'request_id': 'req-cb16c2a6-08b7-457b-a01a-203cadec3c41'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:32.549536 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb16c2a6-08b7-457b-a01a-203cadec3c41 admin admin] The token cache is being invalidate because project 67ed50546d004a459ad43df30992b3d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:37:32.549822 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb16c2a6-08b7-457b-a01a-203cadec3c41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:32.550431 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb16c2a6-08b7-457b-a01a-203cadec3c41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:32.551934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1370/2758] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:37:32 2022] DELETE /identity/v3/projects/67ed50546d004a459ad43df30992b3d2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:37:32.556421 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04d1ec-d9bd-4a73-b4dc-d6ac34e78e4c service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:32.565911 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04d1ec-d9bd-4a73-b4dc-d6ac34e78e4c service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:32.583978 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:32.625962 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d67e4-1c2b-4ef1-bd5b-ed2850ef8c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:32.626842 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d67e4-1c2b-4ef1-bd5b-ed2850ef8c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:32.627131 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d67e4-1c2b-4ef1-bd5b-ed2850ef8c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:32.627415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d67e4-1c2b-4ef1-bd5b-ed2850ef8c1f admin admin] PATH_INFO: `/v3/users/acd0c9a0cce94452a52940cdba26937a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:32.628663 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d67e4-1c2b-4ef1-bd5b-ed2850ef8c1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=acd0c9a0cce94452a52940cdba26937a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:32.630585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d67e4-1c2b-4ef1-bd5b-ed2850ef8c1f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:32.675669 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e31d67e4-1c2b-4ef1-bd5b-ed2850ef8c1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acd0c9a0cce94452a52940cdba26937a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:32.688234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e31d67e4-1c2b-4ef1-bd5b-ed2850ef8c1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acd0c9a0cce94452a52940cdba26937a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:32.688916 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e31d67e4-1c2b-4ef1-bd5b-ed2850ef8c1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acd0c9a0cce94452a52940cdba26937a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:32.689524 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e31d67e4-1c2b-4ef1-bd5b-ed2850ef8c1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acd0c9a0cce94452a52940cdba26937a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:32.697181 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e31d67e4-1c2b-4ef1-bd5b-ed2850ef8c1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acd0c9a0cce94452a52940cdba26937a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:32.709113 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1389/2759] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:37:32 2022] DELETE /identity/v3/users/acd0c9a0cce94452a52940cdba26937a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:37:32.990258 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16c2a6-08b7-457b-a01a-203cadec3c41 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:32.992658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cb16c2a6-08b7-457b-a01a-203cadec3c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:33.017496 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16c2a6-08b7-457b-a01a-203cadec3c41 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:33.065790 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:33.083907 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e269a3-6a80-49ac-993a-24986478b766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:33.084800 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e269a3-6a80-49ac-993a-24986478b766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:33.085087 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e269a3-6a80-49ac-993a-24986478b766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:33.085366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e269a3-6a80-49ac-993a-24986478b766 admin admin] PATH_INFO: `/v3/projects/956c49b590f24ccb8b4b83a5177ba172` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:33.086573 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e269a3-6a80-49ac-993a-24986478b766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=956c49b590f24ccb8b4b83a5177ba172)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:33.088556 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e269a3-6a80-49ac-993a-24986478b766 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:33.134769 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-80e269a3-6a80-49ac-993a-24986478b766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '956c49b590f24ccb8b4b83a5177ba172', 'request_id': 'req-80e269a3-6a80-49ac-993a-24986478b766'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:33.140372 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-80e269a3-6a80-49ac-993a-24986478b766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '956c49b590f24ccb8b4b83a5177ba172', 'request_id': 'req-80e269a3-6a80-49ac-993a-24986478b766'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:33.146531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-80e269a3-6a80-49ac-993a-24986478b766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '956c49b590f24ccb8b4b83a5177ba172', 'request_id': 'req-80e269a3-6a80-49ac-993a-24986478b766'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:33.160884 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-80e269a3-6a80-49ac-993a-24986478b766 admin admin] The token cache is being invalidate because project 956c49b590f24ccb8b4b83a5177ba172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:37:33.161286 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-80e269a3-6a80-49ac-993a-24986478b766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:33.162131 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-80e269a3-6a80-49ac-993a-24986478b766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:37:33.164123 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1371/2760] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:37:32 2022] DELETE /identity/v3/projects/956c49b590f24ccb8b4b83a5177ba172 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:37:35.215333 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d67e4-1c2b-4ef1-bd5b-ed2850ef8c1f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:37:35.216787 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e31d67e4-1c2b-4ef1-bd5b-ed2850ef8c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:35.276035 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d67e4-1c2b-4ef1-bd5b-ed2850ef8c1f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:37:35.328882 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf01c10-d5a3-4a6b-a360-07e6244c4a35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:37:35.329557 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf01c10-d5a3-4a6b-a360-07e6244c4a35 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:35.329838 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf01c10-d5a3-4a6b-a360-07e6244c4a35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:35.330115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf01c10-d5a3-4a6b-a360-07e6244c4a35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:35.331876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-adf01c10-d5a3-4a6b-a360-07e6244c4a35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:35.342668 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:35.393309 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf01c10-d5a3-4a6b-a360-07e6244c4a35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:37:35.394761 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf01c10-d5a3-4a6b-a360-07e6244c4a35 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:37:35.404680 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1390/2761] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:37:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:37:43.157609 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=32,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:44.358502 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be799a6a-1b00-4b6f-9ba4-3c13b0560dbd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:37:44.359137 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be799a6a-1b00-4b6f-9ba4-3c13b0560dbd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:37:44.359259 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be799a6a-1b00-4b6f-9ba4-3c13b0560dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:37:44.379156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-be799a6a-1b00-4b6f-9ba4-3c13b0560dbd None None] MFA Rules not processed for user `4e9eb8d188e040bca894e038955f2402`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:37:44.380526 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-be799a6a-1b00-4b6f-9ba4-3c13b0560dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:37:44.388135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1372/2762] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:37:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:37:45.400109 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=22 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:37:54.374126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:08.154982 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf01c10-d5a3-4a6b-a360-07e6244c4a35 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:08.156481 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-adf01c10-d5a3-4a6b-a360-07e6244c4a35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:08.219882 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf01c10-d5a3-4a6b-a360-07e6244c4a35 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:08.224356 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540ea97-562a-40b8-a54c-f274e4e1febc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:08.225248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540ea97-562a-40b8-a54c-f274e4e1febc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:08.225786 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540ea97-562a-40b8-a54c-f274e4e1febc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:08.226076 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540ea97-562a-40b8-a54c-f274e4e1febc admin admin] PATH_INFO: `/v3/users/794b6ddd679443f584e0903f0216fb64` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:08.227244 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540ea97-562a-40b8-a54c-f274e4e1febc admin admin] RBAC: Authorizing `identity:delete_user(user_id=794b6ddd679443f584e0903f0216fb64)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:08.229334 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540ea97-562a-40b8-a54c-f274e4e1febc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:08.273860 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f540ea97-562a-40b8-a54c-f274e4e1febc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '794b6ddd679443f584e0903f0216fb64'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:38:08.286381 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f540ea97-562a-40b8-a54c-f274e4e1febc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '794b6ddd679443f584e0903f0216fb64'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:38:08.287068 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f540ea97-562a-40b8-a54c-f274e4e1febc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '794b6ddd679443f584e0903f0216fb64'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:38:08.287679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f540ea97-562a-40b8-a54c-f274e4e1febc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '794b6ddd679443f584e0903f0216fb64'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:38:08.294233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f540ea97-562a-40b8-a54c-f274e4e1febc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '794b6ddd679443f584e0903f0216fb64'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:38:08.306284 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1391/2763] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:38:08 2022] DELETE /identity/v3/users/794b6ddd679443f584e0903f0216fb64 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:38:08.311286 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41180b7-5a7e-404c-96c4-68e6c4c95c4e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:08.311799 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41180b7-5a7e-404c-96c4-68e6c4c95c4e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:08.311905 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41180b7-5a7e-404c-96c4-68e6c4c95c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:08.329502 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f41180b7-5a7e-404c-96c4-68e6c4c95c4e None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:08.330704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f41180b7-5a7e-404c-96c4-68e6c4c95c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:08.390772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1373/2764] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:08.401069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540ea97-562a-40b8-a54c-f274e4e1febc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:08.402583 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f540ea97-562a-40b8-a54c-f274e4e1febc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:08.469058 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540ea97-562a-40b8-a54c-f274e4e1febc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:08.472753 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c2ba8-ea73-4187-8d23-adebc6b2c382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:08.473402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c2ba8-ea73-4187-8d23-adebc6b2c382 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:08.473687 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c2ba8-ea73-4187-8d23-adebc6b2c382 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:08.473966 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c2ba8-ea73-4187-8d23-adebc6b2c382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:08.483534 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c2ba8-ea73-4187-8d23-adebc6b2c382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:08.485052 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c2ba8-ea73-4187-8d23-adebc6b2c382 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:08.495212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1392/2765] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:38:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:38:08.784462 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41180b7-5a7e-404c-96c4-68e6c4c95c4e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:08.786290 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f41180b7-5a7e-404c-96c4-68e6c4c95c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:08.801421 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41180b7-5a7e-404c-96c4-68e6c4c95c4e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:08.806216 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3dd47-9431-4ce7-b3e6-75015d3375ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:08.807774 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3dd47-9431-4ce7-b3e6-75015d3375ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:08.808123 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3dd47-9431-4ce7-b3e6-75015d3375ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:08.808404 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3dd47-9431-4ce7-b3e6-75015d3375ac admin admin] PATH_INFO: `/v3/projects/43922a0d85f44e378821e7bd9e956e63` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:08.809803 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3dd47-9431-4ce7-b3e6-75015d3375ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=43922a0d85f44e378821e7bd9e956e63)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:08.811724 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3dd47-9431-4ce7-b3e6-75015d3375ac admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:08.859026 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a2c3dd47-9431-4ce7-b3e6-75015d3375ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43922a0d85f44e378821e7bd9e956e63', 'request_id': 'req-a2c3dd47-9431-4ce7-b3e6-75015d3375ac'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:38:08.864410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a2c3dd47-9431-4ce7-b3e6-75015d3375ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43922a0d85f44e378821e7bd9e956e63', 'request_id': 'req-a2c3dd47-9431-4ce7-b3e6-75015d3375ac'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:38:08.870986 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a2c3dd47-9431-4ce7-b3e6-75015d3375ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43922a0d85f44e378821e7bd9e956e63', 'request_id': 'req-a2c3dd47-9431-4ce7-b3e6-75015d3375ac'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:38:08.883135 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a2c3dd47-9431-4ce7-b3e6-75015d3375ac admin admin] The token cache is being invalidate because project 43922a0d85f44e378821e7bd9e956e63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:38:08.883440 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a2c3dd47-9431-4ce7-b3e6-75015d3375ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:38:08.884073 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a2c3dd47-9431-4ce7-b3e6-75015d3375ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:38:08.885638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1374/2766] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:38:08 2022] DELETE /identity/v3/projects/43922a0d85f44e378821e7bd9e956e63 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:38:18.490384 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:18.880157 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:29.179548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093ef4e-6d8c-4120-9d6a-c2c7fe66120c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:29.180058 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093ef4e-6d8c-4120-9d6a-c2c7fe66120c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:29.180153 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093ef4e-6d8c-4120-9d6a-c2c7fe66120c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:29.208409 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5093ef4e-6d8c-4120-9d6a-c2c7fe66120c None None] MFA Rules not processed for user `a09029144c454b099bea86aaa5229ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:29.209645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5093ef4e-6d8c-4120-9d6a-c2c7fe66120c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:29.271689 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1393/2767] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 18 10:38:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:29.317555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3dd47-9431-4ce7-b3e6-75015d3375ac admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:29.318964 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a2c3dd47-9431-4ce7-b3e6-75015d3375ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:29.388476 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3dd47-9431-4ce7-b3e6-75015d3375ac admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:29.393558 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5189e4-0079-47f3-a29f-cbf805bc3160 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6168873f2846c8af73b903bba17eb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:29.394207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5189e4-0079-47f3-a29f-cbf805bc3160 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:29.394491 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5189e4-0079-47f3-a29f-cbf805bc3160 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:29.394771 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5189e4-0079-47f3-a29f-cbf805bc3160 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:29.403518 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5189e4-0079-47f3-a29f-cbf805bc3160 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:29.405144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5189e4-0079-47f3-a29f-cbf805bc3160 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:29.413217 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1375/2768] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 18 10:38:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:38:39.264880 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=32 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:39.409889 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=32 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:44.194454 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408bcd4-47a4-41b8-9353-9b70631b6395 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:44.194920 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408bcd4-47a4-41b8-9353-9b70631b6395 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:44.195026 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408bcd4-47a4-41b8-9353-9b70631b6395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:44.212433 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4408bcd4-47a4-41b8-9353-9b70631b6395 None None] MFA Rules not processed for user `a09029144c454b099bea86aaa5229ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:38:44.213680 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4408bcd4-47a4-41b8-9353-9b70631b6395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:44.221279 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1394/2769] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jul 18 10:38:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:38:44.233849 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5189e4-0079-47f3-a29f-cbf805bc3160 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:44.244410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5189e4-0079-47f3-a29f-cbf805bc3160 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:44.249497 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72f245-2207-488e-8ad4-9796e4fd23d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6168873f2846c8af73b903bba17eb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:44.250132 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72f245-2207-488e-8ad4-9796e4fd23d9 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:44.250422 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72f245-2207-488e-8ad4-9796e4fd23d9 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:44.250705 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72f245-2207-488e-8ad4-9796e4fd23d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:44.258774 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72f245-2207-488e-8ad4-9796e4fd23d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:44.260361 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72f245-2207-488e-8ad4-9796e4fd23d9 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:44.268955 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1376/2770] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 18 10:38:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:38:54.207287 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:54.220836 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408bcd4-47a4-41b8-9353-9b70631b6395 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:54.222224 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4408bcd4-47a4-41b8-9353-9b70631b6395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:54.265717 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:38:54.277436 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408bcd4-47a4-41b8-9353-9b70631b6395 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:54.281024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d9fa1-e3f1-4125-939e-3ef78a7daf89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:54.281662 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d9fa1-e3f1-4125-939e-3ef78a7daf89 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:54.281947 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d9fa1-e3f1-4125-939e-3ef78a7daf89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:54.282226 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d9fa1-e3f1-4125-939e-3ef78a7daf89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:54.284007 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ea5d9fa1-e3f1-4125-939e-3ef78a7daf89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:54.365838 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d9fa1-e3f1-4125-939e-3ef78a7daf89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:54.367312 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d9fa1-e3f1-4125-939e-3ef78a7daf89 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:54.387913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1395/2771] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:38:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:38:55.560555 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72f245-2207-488e-8ad4-9796e4fd23d9 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:38:55.570315 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72f245-2207-488e-8ad4-9796e4fd23d9 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:38:55.574509 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187a616-3631-4d7d-9d95-f58071cbed2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:38:55.575146 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187a616-3631-4d7d-9d95-f58071cbed2a service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:38:55.575432 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187a616-3631-4d7d-9d95-f58071cbed2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:38:55.575738 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187a616-3631-4d7d-9d95-f58071cbed2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:38:55.577577 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9187a616-3631-4d7d-9d95-f58071cbed2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:38:55.591821 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9187a616-3631-4d7d-9d95-f58071cbed2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:38:55.593269 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9187a616-3631-4d7d-9d95-f58071cbed2a service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:38:55.604805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1377/2772] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:38:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:00.872664 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99391a-daf3-48cb-b11c-6b73732f8ed5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:00.873133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99391a-daf3-48cb-b11c-6b73732f8ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:00.873253 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99391a-daf3-48cb-b11c-6b73732f8ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:00.890633 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0d99391a-daf3-48cb-b11c-6b73732f8ed5 None None] MFA Rules not processed for user `a09029144c454b099bea86aaa5229ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:00.891821 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0d99391a-daf3-48cb-b11c-6b73732f8ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:00.898886 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1396/2773] 10.222.0.33 () {58 vars in 967 bytes} [Mon Jul 18 10:39:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:39:00.911628 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187a616-3631-4d7d-9d95-f58071cbed2a service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:00.923256 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187a616-3631-4d7d-9d95-f58071cbed2a service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:00.928191 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e4be2-1374-47cb-b72c-e9aff9ef14cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6168873f2846c8af73b903bba17eb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:00.928941 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e4be2-1374-47cb-b72c-e9aff9ef14cb service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:00.929226 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e4be2-1374-47cb-b72c-e9aff9ef14cb service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:00.929511 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e4be2-1374-47cb-b72c-e9aff9ef14cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:00.937581 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e4be2-1374-47cb-b72c-e9aff9ef14cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:00.939160 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e4be2-1374-47cb-b72c-e9aff9ef14cb service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:00.947246 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1378/2774] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jul 18 10:39:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 10:39:10.885477 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=66 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:10.943930 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=12 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:27.608910 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99391a-daf3-48cb-b11c-6b73732f8ed5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:27.611073 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0d99391a-daf3-48cb-b11c-6b73732f8ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:27.679975 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99391a-daf3-48cb-b11c-6b73732f8ed5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:27.685328 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001a5af-b047-422e-a4df-b90109efbbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:27.686619 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001a5af-b047-422e-a4df-b90109efbbab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:27.686932 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001a5af-b047-422e-a4df-b90109efbbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:27.687207 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001a5af-b047-422e-a4df-b90109efbbab admin admin] PATH_INFO: `/v3/users/de7190c1ae524da280700ef2548a4a17` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:27.688437 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001a5af-b047-422e-a4df-b90109efbbab admin admin] RBAC: Authorizing `identity:delete_user(user_id=de7190c1ae524da280700ef2548a4a17)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:27.690394 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001a5af-b047-422e-a4df-b90109efbbab admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:27.738675 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f001a5af-b047-422e-a4df-b90109efbbab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de7190c1ae524da280700ef2548a4a17'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:27.751238 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f001a5af-b047-422e-a4df-b90109efbbab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de7190c1ae524da280700ef2548a4a17'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:27.751912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f001a5af-b047-422e-a4df-b90109efbbab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de7190c1ae524da280700ef2548a4a17'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:27.752683 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f001a5af-b047-422e-a4df-b90109efbbab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de7190c1ae524da280700ef2548a4a17'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:27.760623 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f001a5af-b047-422e-a4df-b90109efbbab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de7190c1ae524da280700ef2548a4a17'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:27.772734 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1397/2775] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:39:27 2022] DELETE /identity/v3/users/de7190c1ae524da280700ef2548a4a17 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:39:27.945206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e4be2-1374-47cb-b72c-e9aff9ef14cb service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:27.956371 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e4be2-1374-47cb-b72c-e9aff9ef14cb service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:28.021169 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b01340-c7e9-4d92-8876-c45edba1eb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:28.022023 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b01340-c7e9-4d92-8876-c45edba1eb22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:28.022305 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b01340-c7e9-4d92-8876-c45edba1eb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:28.022576 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b01340-c7e9-4d92-8876-c45edba1eb22 admin admin] PATH_INFO: `/v3/projects/5628452a1ceb4af293ea96c187983fa6` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:28.023742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b01340-c7e9-4d92-8876-c45edba1eb22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5628452a1ceb4af293ea96c187983fa6)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:28.025709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b01340-c7e9-4d92-8876-c45edba1eb22 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:28.071635 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-95b01340-c7e9-4d92-8876-c45edba1eb22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5628452a1ceb4af293ea96c187983fa6', 'request_id': 'req-95b01340-c7e9-4d92-8876-c45edba1eb22'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:28.077045 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-95b01340-c7e9-4d92-8876-c45edba1eb22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5628452a1ceb4af293ea96c187983fa6', 'request_id': 'req-95b01340-c7e9-4d92-8876-c45edba1eb22'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:28.083001 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-95b01340-c7e9-4d92-8876-c45edba1eb22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5628452a1ceb4af293ea96c187983fa6', 'request_id': 'req-95b01340-c7e9-4d92-8876-c45edba1eb22'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:28.094686 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-95b01340-c7e9-4d92-8876-c45edba1eb22 admin admin] The token cache is being invalidate because project 5628452a1ceb4af293ea96c187983fa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:39:28.094977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-95b01340-c7e9-4d92-8876-c45edba1eb22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:28.095587 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-95b01340-c7e9-4d92-8876-c45edba1eb22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:28.097134 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1379/2776] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:39:27 2022] DELETE /identity/v3/projects/5628452a1ceb4af293ea96c187983fa6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:39:28.101246 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001a5af-b047-422e-a4df-b90109efbbab admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:28.102973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f001a5af-b047-422e-a4df-b90109efbbab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:28.116602 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001a5af-b047-422e-a4df-b90109efbbab admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:28.173842 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e5ed9-82e4-4713-895f-6b7b155ea7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:28.174724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e5ed9-82e4-4713-895f-6b7b155ea7ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:28.175016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e5ed9-82e4-4713-895f-6b7b155ea7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:28.175299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e5ed9-82e4-4713-895f-6b7b155ea7ae admin admin] PATH_INFO: `/v3/users/6677d68e1f734f1ca65e0e25f6df3c12` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:28.179357 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e5ed9-82e4-4713-895f-6b7b155ea7ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=6677d68e1f734f1ca65e0e25f6df3c12)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:28.181325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e5ed9-82e4-4713-895f-6b7b155ea7ae admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:28.202862 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:28.228690 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eb0e5ed9-82e4-4713-895f-6b7b155ea7ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6677d68e1f734f1ca65e0e25f6df3c12'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:28.240908 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eb0e5ed9-82e4-4713-895f-6b7b155ea7ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6677d68e1f734f1ca65e0e25f6df3c12'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:28.241639 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eb0e5ed9-82e4-4713-895f-6b7b155ea7ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6677d68e1f734f1ca65e0e25f6df3c12'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:28.242304 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eb0e5ed9-82e4-4713-895f-6b7b155ea7ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6677d68e1f734f1ca65e0e25f6df3c12'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:28.249625 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-eb0e5ed9-82e4-4713-895f-6b7b155ea7ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6677d68e1f734f1ca65e0e25f6df3c12'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:28.261815 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1398/2777] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:39:28 2022] DELETE /identity/v3/users/6677d68e1f734f1ca65e0e25f6df3c12 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:39:28.430949 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b01340-c7e9-4d92-8876-c45edba1eb22 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:28.432600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-95b01340-c7e9-4d92-8876-c45edba1eb22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:28.450018 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b01340-c7e9-4d92-8876-c45edba1eb22 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:28.507702 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49be282-c524-449a-ab2a-bf5a4489dcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:28.508579 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49be282-c524-449a-ab2a-bf5a4489dcea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:28.508861 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49be282-c524-449a-ab2a-bf5a4489dcea admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:28.509132 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49be282-c524-449a-ab2a-bf5a4489dcea admin admin] PATH_INFO: `/v3/projects/a288f00152ee4a52945eb7be54d28207` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:28.510298 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49be282-c524-449a-ab2a-bf5a4489dcea admin admin] RBAC: Authorizing `identity:delete_project(project_id=a288f00152ee4a52945eb7be54d28207)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:28.512188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49be282-c524-449a-ab2a-bf5a4489dcea admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:28.546764 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:28.560664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f49be282-c524-449a-ab2a-bf5a4489dcea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a288f00152ee4a52945eb7be54d28207', 'request_id': 'req-f49be282-c524-449a-ab2a-bf5a4489dcea'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:28.566173 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f49be282-c524-449a-ab2a-bf5a4489dcea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a288f00152ee4a52945eb7be54d28207', 'request_id': 'req-f49be282-c524-449a-ab2a-bf5a4489dcea'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:28.572180 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f49be282-c524-449a-ab2a-bf5a4489dcea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a288f00152ee4a52945eb7be54d28207', 'request_id': 'req-f49be282-c524-449a-ab2a-bf5a4489dcea'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:28.585351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f49be282-c524-449a-ab2a-bf5a4489dcea admin admin] The token cache is being invalidate because project a288f00152ee4a52945eb7be54d28207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:39:28.585643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f49be282-c524-449a-ab2a-bf5a4489dcea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:28.586282 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f49be282-c524-449a-ab2a-bf5a4489dcea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:28.587845 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1380/2778] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:39:28 2022] DELETE /identity/v3/projects/a288f00152ee4a52945eb7be54d28207 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:39:34.803138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e5ed9-82e4-4713-895f-6b7b155ea7ae admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:34.804587 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-eb0e5ed9-82e4-4713-895f-6b7b155ea7ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:34.860796 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e5ed9-82e4-4713-895f-6b7b155ea7ae admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:34.910207 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844cc02-1b1f-4480-89a8-72dd58e69e54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:34.910852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844cc02-1b1f-4480-89a8-72dd58e69e54 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:34.911165 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844cc02-1b1f-4480-89a8-72dd58e69e54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:34.911438 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844cc02-1b1f-4480-89a8-72dd58e69e54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:34.913194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0844cc02-1b1f-4480-89a8-72dd58e69e54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:34.970405 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844cc02-1b1f-4480-89a8-72dd58e69e54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:34.971846 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844cc02-1b1f-4480-89a8-72dd58e69e54 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:34.981023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1399/2779] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:37.567109 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49be282-c524-449a-ab2a-bf5a4489dcea admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:37.570303 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f49be282-c524-449a-ab2a-bf5a4489dcea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:37.706751 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49be282-c524-449a-ab2a-bf5a4489dcea admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:37.726511 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27095ce8-1861-4300-8322-53024af59663 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:37.727647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27095ce8-1861-4300-8322-53024af59663 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:37.727647 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27095ce8-1861-4300-8322-53024af59663 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:37.728139 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27095ce8-1861-4300-8322-53024af59663 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:37.735499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-27095ce8-1861-4300-8322-53024af59663 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:37.751132 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27095ce8-1861-4300-8322-53024af59663 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:37.753648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27095ce8-1861-4300-8322-53024af59663 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:37.779594 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1381/2780] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:41.844432 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844cc02-1b1f-4480-89a8-72dd58e69e54 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:41.845881 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0844cc02-1b1f-4480-89a8-72dd58e69e54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:41.856832 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:41.903602 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844cc02-1b1f-4480-89a8-72dd58e69e54 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:41.907712 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06be44-8a92-469a-84b6-6cfb59cc5cec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:41.908351 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06be44-8a92-469a-84b6-6cfb59cc5cec service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:41.908682 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06be44-8a92-469a-84b6-6cfb59cc5cec service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:41.908959 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06be44-8a92-469a-84b6-6cfb59cc5cec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:41.910726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dc06be44-8a92-469a-84b6-6cfb59cc5cec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:41.972455 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06be44-8a92-469a-84b6-6cfb59cc5cec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:41.973939 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06be44-8a92-469a-84b6-6cfb59cc5cec service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:41.983659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1400/2781] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:45.581060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27095ce8-1861-4300-8322-53024af59663 service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:45.582546 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-27095ce8-1861-4300-8322-53024af59663 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:45.596953 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27095ce8-1861-4300-8322-53024af59663 service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:45.601482 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5a321-1c65-40f0-b0b0-311e0b778b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:45.602364 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5a321-1c65-40f0-b0b0-311e0b778b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:45.602648 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5a321-1c65-40f0-b0b0-311e0b778b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:45.602925 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5a321-1c65-40f0-b0b0-311e0b778b5c admin admin] PATH_INFO: `/v3/users/cacc328f0f3540299310a0e53afe6acb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:45.604447 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5a321-1c65-40f0-b0b0-311e0b778b5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cacc328f0f3540299310a0e53afe6acb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:45.606430 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5a321-1c65-40f0-b0b0-311e0b778b5c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:45.656888 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-68a5a321-1c65-40f0-b0b0-311e0b778b5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cacc328f0f3540299310a0e53afe6acb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:45.657473 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-68a5a321-1c65-40f0-b0b0-311e0b778b5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cacc328f0f3540299310a0e53afe6acb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:45.658088 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-68a5a321-1c65-40f0-b0b0-311e0b778b5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cacc328f0f3540299310a0e53afe6acb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:45.664908 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-68a5a321-1c65-40f0-b0b0-311e0b778b5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cacc328f0f3540299310a0e53afe6acb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:45.677590 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-68a5a321-1c65-40f0-b0b0-311e0b778b5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cacc328f0f3540299310a0e53afe6acb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:45.690132 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1382/2782] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:39:45 2022] DELETE /identity/v3/users/cacc328f0f3540299310a0e53afe6acb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:39:45.875599 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06be44-8a92-469a-84b6-6cfb59cc5cec service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:45.899093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06be44-8a92-469a-84b6-6cfb59cc5cec service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:45.955817 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5a321-1c65-40f0-b0b0-311e0b778b5c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:45.958043 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-68a5a321-1c65-40f0-b0b0-311e0b778b5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:45.984802 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:46.038514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8d072-09b8-4635-a96d-a13aabb79241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:46.040056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8d072-09b8-4635-a96d-a13aabb79241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:46.040750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8d072-09b8-4635-a96d-a13aabb79241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:46.042365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8d072-09b8-4635-a96d-a13aabb79241 admin admin] PATH_INFO: `/v3/projects/feefb94984d54cbbaa70b963d956a520` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:46.044187 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8d072-09b8-4635-a96d-a13aabb79241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=feefb94984d54cbbaa70b963d956a520)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:46.046309 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5a321-1c65-40f0-b0b0-311e0b778b5c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:46.047973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8d072-09b8-4635-a96d-a13aabb79241 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:46.051028 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42069ae-b41b-491c-8cbd-893671a898fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:46.052197 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42069ae-b41b-491c-8cbd-893671a898fb service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:46.052587 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42069ae-b41b-491c-8cbd-893671a898fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:46.052942 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42069ae-b41b-491c-8cbd-893671a898fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:46.055291 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d42069ae-b41b-491c-8cbd-893671a898fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:46.068646 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42069ae-b41b-491c-8cbd-893671a898fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:46.070253 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42069ae-b41b-491c-8cbd-893671a898fb service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:46.081230 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1383/2783] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:39:46.124217 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-bff8d072-09b8-4635-a96d-a13aabb79241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feefb94984d54cbbaa70b963d956a520', 'request_id': 'req-bff8d072-09b8-4635-a96d-a13aabb79241'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:46.132666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-bff8d072-09b8-4635-a96d-a13aabb79241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feefb94984d54cbbaa70b963d956a520', 'request_id': 'req-bff8d072-09b8-4635-a96d-a13aabb79241'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:46.136719 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:46.142306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-bff8d072-09b8-4635-a96d-a13aabb79241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feefb94984d54cbbaa70b963d956a520', 'request_id': 'req-bff8d072-09b8-4635-a96d-a13aabb79241'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:46.164117 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-bff8d072-09b8-4635-a96d-a13aabb79241 admin admin] The token cache is being invalidate because project feefb94984d54cbbaa70b963d956a520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:39:46.164402 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-bff8d072-09b8-4635-a96d-a13aabb79241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:46.165023 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-bff8d072-09b8-4635-a96d-a13aabb79241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:46.166574 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1401/2784] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:39:45 2022] DELETE /identity/v3/projects/feefb94984d54cbbaa70b963d956a520 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:39:46.171744 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad755e5-250e-4bcc-b67c-4d77ebe7abde None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:46.172060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad755e5-250e-4bcc-b67c-4d77ebe7abde None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:46.172345 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad755e5-250e-4bcc-b67c-4d77ebe7abde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:46.197798 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bad755e5-250e-4bcc-b67c-4d77ebe7abde None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:46.199011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bad755e5-250e-4bcc-b67c-4d77ebe7abde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:46.261188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1384/2785] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:39:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:46.269619 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8141c-1a12-433b-aaab-f2696e0741cb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:46.269924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8141c-1a12-433b-aaab-f2696e0741cb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:46.270210 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8141c-1a12-433b-aaab-f2696e0741cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:46.289124 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ced8141c-1a12-433b-aaab-f2696e0741cb None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:46.290340 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ced8141c-1a12-433b-aaab-f2696e0741cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:46.298547 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1402/2786] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:39:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:46.302574 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad755e5-250e-4bcc-b67c-4d77ebe7abde None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:46.312826 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad755e5-250e-4bcc-b67c-4d77ebe7abde None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:46.317456 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ed8d1-e28c-473a-8ebf-2dd381edc1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:46.318111 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ed8d1-e28c-473a-8ebf-2dd381edc1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:46.318397 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ed8d1-e28c-473a-8ebf-2dd381edc1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:46.318675 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ed8d1-e28c-473a-8ebf-2dd381edc1d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:46.319461 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ed8d1-e28c-473a-8ebf-2dd381edc1d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:39:46.319779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ed8d1-e28c-473a-8ebf-2dd381edc1d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:46.321413 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ed8d1-e28c-473a-8ebf-2dd381edc1d5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:46.337031 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1385/2787] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:39:46.338132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8141c-1a12-433b-aaab-f2696e0741cb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:46.347692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8141c-1a12-433b-aaab-f2696e0741cb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:46.352475 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447c71e-2444-47a2-bed1-fdcc30006b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:46.353446 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447c71e-2444-47a2-bed1-fdcc30006b4a admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:46.353769 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447c71e-2444-47a2-bed1-fdcc30006b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:46.354051 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447c71e-2444-47a2-bed1-fdcc30006b4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:46.354791 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447c71e-2444-47a2-bed1-fdcc30006b4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:46.356749 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447c71e-2444-47a2-bed1-fdcc30006b4a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:46.397514 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1403/2788] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:39:46 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:39:46.398643 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ed8d1-e28c-473a-8ebf-2dd381edc1d5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:46.408620 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ed8d1-e28c-473a-8ebf-2dd381edc1d5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:46.466846 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bb725-e9f5-46f5-b437-9c1d3ebebda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:46.467417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bb725-e9f5-46f5-b437-9c1d3ebebda2 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:46.467702 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bb725-e9f5-46f5-b437-9c1d3ebebda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:46.467980 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bb725-e9f5-46f5-b437-9c1d3ebebda2 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:46.468716 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bb725-e9f5-46f5-b437-9c1d3ebebda2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:46.470649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bb725-e9f5-46f5-b437-9c1d3ebebda2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:46.492017 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1386/2789] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:39:46 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:39:46.496355 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447c71e-2444-47a2-bed1-fdcc30006b4a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:46.507912 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447c71e-2444-47a2-bed1-fdcc30006b4a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:46.512536 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9b95f-f6b9-4a01-ae02-076a7d0f1435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:46.513203 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9b95f-f6b9-4a01-ae02-076a7d0f1435 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:46.513486 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9b95f-f6b9-4a01-ae02-076a7d0f1435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:46.513760 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9b95f-f6b9-4a01-ae02-076a7d0f1435 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:46.514484 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9b95f-f6b9-4a01-ae02-076a7d0f1435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:46.516085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9b95f-f6b9-4a01-ae02-076a7d0f1435 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:46.528994 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1404/2790] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:39:46.530147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bb725-e9f5-46f5-b437-9c1d3ebebda2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:46.540011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bb725-e9f5-46f5-b437-9c1d3ebebda2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:46.544557 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21148c52-9ee4-423d-801c-c8fdc299aae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:46.545722 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21148c52-9ee4-423d-801c-c8fdc299aae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:46.546006 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21148c52-9ee4-423d-801c-c8fdc299aae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:46.546280 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21148c52-9ee4-423d-801c-c8fdc299aae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:46.547002 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21148c52-9ee4-423d-801c-c8fdc299aae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:46.548632 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21148c52-9ee4-423d-801c-c8fdc299aae6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:46.562233 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1387/2791] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:39:46.563448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9b95f-f6b9-4a01-ae02-076a7d0f1435 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:46.573760 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9b95f-f6b9-4a01-ae02-076a7d0f1435 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:46.578558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde0651-774b-444f-b7fc-5a043407d6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:46.579521 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde0651-774b-444f-b7fc-5a043407d6ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:46.579818 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde0651-774b-444f-b7fc-5a043407d6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:46.580085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde0651-774b-444f-b7fc-5a043407d6ab admin admin] PATH_INFO: `/v3/projects/9d61432080064383bc2629af75576900/users/5b7396918821425392a2c93d5191ec7c/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:46.596569 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde0651-774b-444f-b7fc-5a043407d6ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d61432080064383bc2629af75576900, user_id=5b7396918821425392a2c93d5191ec7c, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:46.599294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde0651-774b-444f-b7fc-5a043407d6ab admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:46.614093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1405/2792] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:39:46 2022] PUT /identity/v3/projects/9d61432080064383bc2629af75576900/users/5b7396918821425392a2c93d5191ec7c/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:39:46.624563 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa76a7-bbaf-49d0-926a-0fd1aca4a28a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:46.625084 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa76a7-bbaf-49d0-926a-0fd1aca4a28a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:46.625187 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa76a7-bbaf-49d0-926a-0fd1aca4a28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:46.652468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-05aa76a7-bbaf-49d0-926a-0fd1aca4a28a None None] MFA Rules not processed for user `5b7396918821425392a2c93d5191ec7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:46.653844 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-05aa76a7-bbaf-49d0-926a-0fd1aca4a28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:46.666842 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:46.721139 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1388/2793] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:39:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:46.733399 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde0651-774b-444f-b7fc-5a043407d6ab admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:46.735015 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dfde0651-774b-444f-b7fc-5a043407d6ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:46.813764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde0651-774b-444f-b7fc-5a043407d6ab admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:46.822056 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98791bf-0431-4173-b9eb-d7400648c9ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29048c2021e340b081ae44a390583e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:46.823240 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98791bf-0431-4173-b9eb-d7400648c9ed service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:46.824445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98791bf-0431-4173-b9eb-d7400648c9ed service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:46.824445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98791bf-0431-4173-b9eb-d7400648c9ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:46.838631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98791bf-0431-4173-b9eb-d7400648c9ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:46.840278 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98791bf-0431-4173-b9eb-d7400648c9ed service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:46.849948 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1406/2794] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:47.072905 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa76a7-bbaf-49d0-926a-0fd1aca4a28a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:47.134967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa76a7-bbaf-49d0-926a-0fd1aca4a28a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:47.139474 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1969af-c204-422d-8250-20585cf9d6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:47.140330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1969af-c204-422d-8250-20585cf9d6b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:47.140644 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1969af-c204-422d-8250-20585cf9d6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:47.140923 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1969af-c204-422d-8250-20585cf9d6b0 admin admin] PATH_INFO: `/v3/users/5b7396918821425392a2c93d5191ec7c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:47.142109 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1969af-c204-422d-8250-20585cf9d6b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b7396918821425392a2c93d5191ec7c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:47.144049 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1969af-c204-422d-8250-20585cf9d6b0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:47.194661 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3a1969af-c204-422d-8250-20585cf9d6b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b7396918821425392a2c93d5191ec7c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:47.195336 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3a1969af-c204-422d-8250-20585cf9d6b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b7396918821425392a2c93d5191ec7c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:47.195971 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3a1969af-c204-422d-8250-20585cf9d6b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b7396918821425392a2c93d5191ec7c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:47.203562 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3a1969af-c204-422d-8250-20585cf9d6b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b7396918821425392a2c93d5191ec7c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:47.215571 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3a1969af-c204-422d-8250-20585cf9d6b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b7396918821425392a2c93d5191ec7c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:47.231487 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1389/2795] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:39:47 2022] DELETE /identity/v3/users/5b7396918821425392a2c93d5191ec7c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:39:47.236161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf062cc-53cf-4cf6-9b9e-97918e3364ff None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:47.236694 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf062cc-53cf-4cf6-9b9e-97918e3364ff None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:47.237212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf062cc-53cf-4cf6-9b9e-97918e3364ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:47.254688 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8bf062cc-53cf-4cf6-9b9e-97918e3364ff None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:47.256067 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8bf062cc-53cf-4cf6-9b9e-97918e3364ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:47.331849 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1407/2796] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:47.341627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1969af-c204-422d-8250-20585cf9d6b0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:47.343662 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3a1969af-c204-422d-8250-20585cf9d6b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:47.384802 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:47.413474 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1969af-c204-422d-8250-20585cf9d6b0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:47.417261 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58018b51-f28c-4321-a16c-1c925be7a189 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:47.417913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58018b51-f28c-4321-a16c-1c925be7a189 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:47.418199 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58018b51-f28c-4321-a16c-1c925be7a189 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:47.418477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58018b51-f28c-4321-a16c-1c925be7a189 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:47.427856 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58018b51-f28c-4321-a16c-1c925be7a189 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:47.429336 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58018b51-f28c-4321-a16c-1c925be7a189 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:47.439318 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1390/2797] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:39:47.866348 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf062cc-53cf-4cf6-9b9e-97918e3364ff None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:47.867883 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8bf062cc-53cf-4cf6-9b9e-97918e3364ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:47.875539 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:47.882094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf062cc-53cf-4cf6-9b9e-97918e3364ff None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:47.887833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605ad4b-7c32-441c-8aa1-bb44a1967e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:47.889775 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605ad4b-7c32-441c-8aa1-bb44a1967e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:47.890234 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605ad4b-7c32-441c-8aa1-bb44a1967e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:47.891414 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605ad4b-7c32-441c-8aa1-bb44a1967e8c admin admin] PATH_INFO: `/v3/projects/9d61432080064383bc2629af75576900` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:47.893130 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605ad4b-7c32-441c-8aa1-bb44a1967e8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d61432080064383bc2629af75576900)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:47.895944 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605ad4b-7c32-441c-8aa1-bb44a1967e8c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:47.949147 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a605ad4b-7c32-441c-8aa1-bb44a1967e8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d61432080064383bc2629af75576900', 'request_id': 'req-a605ad4b-7c32-441c-8aa1-bb44a1967e8c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:47.955318 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a605ad4b-7c32-441c-8aa1-bb44a1967e8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d61432080064383bc2629af75576900', 'request_id': 'req-a605ad4b-7c32-441c-8aa1-bb44a1967e8c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:47.961542 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a605ad4b-7c32-441c-8aa1-bb44a1967e8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d61432080064383bc2629af75576900', 'request_id': 'req-a605ad4b-7c32-441c-8aa1-bb44a1967e8c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:47.974704 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a605ad4b-7c32-441c-8aa1-bb44a1967e8c admin admin] The token cache is being invalidate because project 9d61432080064383bc2629af75576900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:39:47.975002 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a605ad4b-7c32-441c-8aa1-bb44a1967e8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:47.975650 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a605ad4b-7c32-441c-8aa1-bb44a1967e8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:47.977688 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1408/2798] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:39:47 2022] DELETE /identity/v3/projects/9d61432080064383bc2629af75576900 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:39:47.983507 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447931b-46a6-4a40-b5a3-6f587e11a26f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:47.984104 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447931b-46a6-4a40-b5a3-6f587e11a26f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:47.984104 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447931b-46a6-4a40-b5a3-6f587e11a26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:48.002709 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3447931b-46a6-4a40-b5a3-6f587e11a26f None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:48.003915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3447931b-46a6-4a40-b5a3-6f587e11a26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:48.063115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1391/2799] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:48.071082 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d367ce-54b7-432e-ae25-eb8032499b97 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:48.071303 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d367ce-54b7-432e-ae25-eb8032499b97 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:48.071581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d367ce-54b7-432e-ae25-eb8032499b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:48.089827 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c2d367ce-54b7-432e-ae25-eb8032499b97 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:48.091043 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c2d367ce-54b7-432e-ae25-eb8032499b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:48.099263 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1409/2800] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:48.103759 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447931b-46a6-4a40-b5a3-6f587e11a26f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:48.114308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447931b-46a6-4a40-b5a3-6f587e11a26f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:48.119860 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943dce1-8b58-48ee-9021-95e06850ea38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:48.120572 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943dce1-8b58-48ee-9021-95e06850ea38 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:48.120863 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943dce1-8b58-48ee-9021-95e06850ea38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:48.121144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943dce1-8b58-48ee-9021-95e06850ea38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:48.121890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943dce1-8b58-48ee-9021-95e06850ea38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:39:48.122203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943dce1-8b58-48ee-9021-95e06850ea38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:48.124532 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943dce1-8b58-48ee-9021-95e06850ea38 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:48.139811 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1392/2801] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:39:48.142028 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d367ce-54b7-432e-ae25-eb8032499b97 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:48.151423 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d367ce-54b7-432e-ae25-eb8032499b97 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:48.156319 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518657e-97bc-4bde-b8bf-824bcef73136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:48.157042 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518657e-97bc-4bde-b8bf-824bcef73136 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:48.157291 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518657e-97bc-4bde-b8bf-824bcef73136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:48.157568 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518657e-97bc-4bde-b8bf-824bcef73136 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:48.158297 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518657e-97bc-4bde-b8bf-824bcef73136 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:48.160212 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518657e-97bc-4bde-b8bf-824bcef73136 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:48.192154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1410/2802] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:39:48 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:39:48.196378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943dce1-8b58-48ee-9021-95e06850ea38 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:48.207531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943dce1-8b58-48ee-9021-95e06850ea38 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:48.267956 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca39f0-975e-415d-95f9-13ad27ded8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:48.268671 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca39f0-975e-415d-95f9-13ad27ded8ea admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:48.268958 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca39f0-975e-415d-95f9-13ad27ded8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:48.269233 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca39f0-975e-415d-95f9-13ad27ded8ea admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:48.269936 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca39f0-975e-415d-95f9-13ad27ded8ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:48.271853 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca39f0-975e-415d-95f9-13ad27ded8ea admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:48.295627 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1393/2803] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:39:48 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:39:48.296645 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518657e-97bc-4bde-b8bf-824bcef73136 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:48.308160 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518657e-97bc-4bde-b8bf-824bcef73136 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:48.317044 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f152b-1b9e-42f4-a942-c63d65233678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:48.317900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f152b-1b9e-42f4-a942-c63d65233678 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:48.318245 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f152b-1b9e-42f4-a942-c63d65233678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:48.318586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f152b-1b9e-42f4-a942-c63d65233678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:48.319431 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f152b-1b9e-42f4-a942-c63d65233678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:48.321211 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f152b-1b9e-42f4-a942-c63d65233678 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:48.332910 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1411/2804] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:39:48.337075 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca39f0-975e-415d-95f9-13ad27ded8ea admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:48.348189 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca39f0-975e-415d-95f9-13ad27ded8ea admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:48.353704 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d5654-ca94-414a-8d6a-c2d0c1fb8dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:48.354404 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d5654-ca94-414a-8d6a-c2d0c1fb8dce admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:48.354658 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d5654-ca94-414a-8d6a-c2d0c1fb8dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:48.354934 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d5654-ca94-414a-8d6a-c2d0c1fb8dce admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:48.355895 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d5654-ca94-414a-8d6a-c2d0c1fb8dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:48.358328 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d5654-ca94-414a-8d6a-c2d0c1fb8dce admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:48.369330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1394/2805] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:39:48.373430 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f152b-1b9e-42f4-a942-c63d65233678 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:48.386859 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f152b-1b9e-42f4-a942-c63d65233678 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:48.391842 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8b494-2503-4045-8a22-0c6db1768edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:48.392765 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8b494-2503-4045-8a22-0c6db1768edc admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:48.393055 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8b494-2503-4045-8a22-0c6db1768edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:48.393337 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8b494-2503-4045-8a22-0c6db1768edc admin admin] PATH_INFO: `/v3/projects/661fd65930ca4599813ca34bf91e2b08/users/1ab1b221dcb4470cb4b063eae5a2021d/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:48.411844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8b494-2503-4045-8a22-0c6db1768edc admin admin] RBAC: Authorizing `identity:create_grant(project_id=661fd65930ca4599813ca34bf91e2b08, user_id=1ab1b221dcb4470cb4b063eae5a2021d, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:48.414554 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8b494-2503-4045-8a22-0c6db1768edc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:48.432935 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1412/2806] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:39:48 2022] PUT /identity/v3/projects/661fd65930ca4599813ca34bf91e2b08/users/1ab1b221dcb4470cb4b063eae5a2021d/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:39:48.444335 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f52ef2-64d9-47b4-b732-a2506c6f97e1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:48.444665 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f52ef2-64d9-47b4-b732-a2506c6f97e1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:48.444946 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f52ef2-64d9-47b4-b732-a2506c6f97e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:48.473219 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b6f52ef2-64d9-47b4-b732-a2506c6f97e1 None None] MFA Rules not processed for user `1ab1b221dcb4470cb4b063eae5a2021d`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:48.474618 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b6f52ef2-64d9-47b4-b732-a2506c6f97e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:48.515347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:48.549089 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1395/2807] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:39:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:48.553564 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8b494-2503-4045-8a22-0c6db1768edc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:48.617452 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8b494-2503-4045-8a22-0c6db1768edc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:48.622616 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046efe96-e7d0-44b2-84fd-e0a5c9168612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:48.623814 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046efe96-e7d0-44b2-84fd-e0a5c9168612 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:48.624099 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046efe96-e7d0-44b2-84fd-e0a5c9168612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:48.624433 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046efe96-e7d0-44b2-84fd-e0a5c9168612 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:48.625164 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046efe96-e7d0-44b2-84fd-e0a5c9168612 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:48.627096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046efe96-e7d0-44b2-84fd-e0a5c9168612 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:48.655878 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1413/2808] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:39:48 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:39:48.659727 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f52ef2-64d9-47b4-b732-a2506c6f97e1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:48.675809 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f52ef2-64d9-47b4-b732-a2506c6f97e1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:48.740659 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd6fb8-2147-4469-846f-fdc5779c5f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:48.741351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd6fb8-2147-4469-846f-fdc5779c5f1a admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:48.741631 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd6fb8-2147-4469-846f-fdc5779c5f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:48.741907 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd6fb8-2147-4469-846f-fdc5779c5f1a admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:48.742622 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd6fb8-2147-4469-846f-fdc5779c5f1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:48.744584 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd6fb8-2147-4469-846f-fdc5779c5f1a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:48.766839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1396/2809] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:39:48 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:39:48.770833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046efe96-e7d0-44b2-84fd-e0a5c9168612 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:48.782210 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046efe96-e7d0-44b2-84fd-e0a5c9168612 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:48.787001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8026e14-ff77-49fc-8695-8df813dd8697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:48.787899 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8026e14-ff77-49fc-8695-8df813dd8697 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:48.788193 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8026e14-ff77-49fc-8695-8df813dd8697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:48.788472 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8026e14-ff77-49fc-8695-8df813dd8697 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:48.789244 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8026e14-ff77-49fc-8695-8df813dd8697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:48.790868 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8026e14-ff77-49fc-8695-8df813dd8697 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:48.793796 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:48.810284 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1414/2810] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:39:48.811239 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd6fb8-2147-4469-846f-fdc5779c5f1a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:48.821554 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd6fb8-2147-4469-846f-fdc5779c5f1a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:48.826278 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a1348-a65f-414c-b68e-4e9021c8c4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:48.827170 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a1348-a65f-414c-b68e-4e9021c8c4e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:48.827460 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a1348-a65f-414c-b68e-4e9021c8c4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:48.827742 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a1348-a65f-414c-b68e-4e9021c8c4e8 admin admin] PATH_INFO: `/v3/projects/245bc6b2a4524a4ebe0cab7d1c561fb0/users/18e0df3304254c36864cd3ed685ba635/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:48.842324 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a1348-a65f-414c-b68e-4e9021c8c4e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=245bc6b2a4524a4ebe0cab7d1c561fb0, user_id=18e0df3304254c36864cd3ed685ba635, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:48.845095 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a1348-a65f-414c-b68e-4e9021c8c4e8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:48.858721 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1397/2811] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:39:48 2022] PUT /identity/v3/projects/245bc6b2a4524a4ebe0cab7d1c561fb0/users/18e0df3304254c36864cd3ed685ba635/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:39:48.862787 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8026e14-ff77-49fc-8695-8df813dd8697 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:48.872825 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8026e14-ff77-49fc-8695-8df813dd8697 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:48.928379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f42b9-81f8-4f38-a08e-d69b53fdaad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:48.929013 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f42b9-81f8-4f38-a08e-d69b53fdaad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:48.929297 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f42b9-81f8-4f38-a08e-d69b53fdaad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:48.929575 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f42b9-81f8-4f38-a08e-d69b53fdaad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:48.930300 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f42b9-81f8-4f38-a08e-d69b53fdaad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:48.931894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f42b9-81f8-4f38-a08e-d69b53fdaad3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:48.942146 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1415/2812] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:39:48.946398 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a1348-a65f-414c-b68e-4e9021c8c4e8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:48.956549 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a1348-a65f-414c-b68e-4e9021c8c4e8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:48.961080 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f4cb1-0d4a-4825-9d0a-c591df26bc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:48.961953 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f4cb1-0d4a-4825-9d0a-c591df26bc8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:48.962237 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f4cb1-0d4a-4825-9d0a-c591df26bc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:48.962519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f4cb1-0d4a-4825-9d0a-c591df26bc8b admin admin] PATH_INFO: `/v3/projects/245bc6b2a4524a4ebe0cab7d1c561fb0/users/18e0df3304254c36864cd3ed685ba635/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:48.964477 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f4cb1-0d4a-4825-9d0a-c591df26bc8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=245bc6b2a4524a4ebe0cab7d1c561fb0, user_id=18e0df3304254c36864cd3ed685ba635, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:48.967188 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f4cb1-0d4a-4825-9d0a-c591df26bc8b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:48.977309 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1398/2813] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:39:48 2022] PUT /identity/v3/projects/245bc6b2a4524a4ebe0cab7d1c561fb0/users/18e0df3304254c36864cd3ed685ba635/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:39:48.988777 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f4097-0f97-48b6-b279-2839e9cc8ca6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:48.989415 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f4097-0f97-48b6-b279-2839e9cc8ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:48.990033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f4097-0f97-48b6-b279-2839e9cc8ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:49.018247 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d87f4097-0f97-48b6-b279-2839e9cc8ca6 None None] MFA Rules not processed for user `18e0df3304254c36864cd3ed685ba635`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:49.019466 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d87f4097-0f97-48b6-b279-2839e9cc8ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:49.089085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1416/2814] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:39:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:49.101543 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f4cb1-0d4a-4825-9d0a-c591df26bc8b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:49.104047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ea0f4cb1-0d4a-4825-9d0a-c591df26bc8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:49.169211 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f4cb1-0d4a-4825-9d0a-c591df26bc8b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:49.173547 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aace511-4130-42d4-abd3-9565951adf3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:49.174212 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aace511-4130-42d4-abd3-9565951adf3d service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:49.174499 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aace511-4130-42d4-abd3-9565951adf3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:49.174773 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aace511-4130-42d4-abd3-9565951adf3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:49.184436 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aace511-4130-42d4-abd3-9565951adf3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:49.185972 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aace511-4130-42d4-abd3-9565951adf3d service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:49.195548 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1399/2815] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:49 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:49.264248 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f4097-0f97-48b6-b279-2839e9cc8ca6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:49.273804 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f4097-0f97-48b6-b279-2839e9cc8ca6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:49.277352 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef9b02-f2af-4130-9fbd-9a612736cae6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:49.278017 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef9b02-f2af-4130-9fbd-9a612736cae6 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:49.278305 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef9b02-f2af-4130-9fbd-9a612736cae6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:49.278581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef9b02-f2af-4130-9fbd-9a612736cae6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:49.336058 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef9b02-f2af-4130-9fbd-9a612736cae6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:49.337548 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef9b02-f2af-4130-9fbd-9a612736cae6 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:49.346997 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1417/2816] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:58.145901 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aace511-4130-42d4-abd3-9565951adf3d service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:58.176579 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:58.209804 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aace511-4130-42d4-abd3-9565951adf3d service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:58.214370 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99fe58-ccb7-47a6-899b-2ee0cb54f0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:58.215235 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99fe58-ccb7-47a6-899b-2ee0cb54f0d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:58.215551 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99fe58-ccb7-47a6-899b-2ee0cb54f0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:58.215836 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99fe58-ccb7-47a6-899b-2ee0cb54f0d9 admin admin] PATH_INFO: `/v3/users/1ab1b221dcb4470cb4b063eae5a2021d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:58.217063 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99fe58-ccb7-47a6-899b-2ee0cb54f0d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ab1b221dcb4470cb4b063eae5a2021d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:58.219177 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99fe58-ccb7-47a6-899b-2ee0cb54f0d9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:58.269034 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6f99fe58-ccb7-47a6-899b-2ee0cb54f0d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ab1b221dcb4470cb4b063eae5a2021d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:58.269834 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6f99fe58-ccb7-47a6-899b-2ee0cb54f0d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ab1b221dcb4470cb4b063eae5a2021d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:58.270773 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6f99fe58-ccb7-47a6-899b-2ee0cb54f0d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ab1b221dcb4470cb4b063eae5a2021d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:58.277640 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6f99fe58-ccb7-47a6-899b-2ee0cb54f0d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ab1b221dcb4470cb4b063eae5a2021d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:58.290067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-6f99fe58-ccb7-47a6-899b-2ee0cb54f0d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ab1b221dcb4470cb4b063eae5a2021d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:58.303543 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1400/2817] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:39:58 2022] DELETE /identity/v3/users/1ab1b221dcb4470cb4b063eae5a2021d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:39:58.307964 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50d192-e0c7-47a5-81f3-6d5e6493b3e4 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:58.308534 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50d192-e0c7-47a5-81f3-6d5e6493b3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:58.308534 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50d192-e0c7-47a5-81f3-6d5e6493b3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:58.325407 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-aa50d192-e0c7-47a5-81f3-6d5e6493b3e4 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:58.326590 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-aa50d192-e0c7-47a5-81f3-6d5e6493b3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:58.335937 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:58.385680 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1418/2818] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:39:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:58.395917 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99fe58-ccb7-47a6-899b-2ee0cb54f0d9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:58.397325 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6f99fe58-ccb7-47a6-899b-2ee0cb54f0d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:58.454189 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99fe58-ccb7-47a6-899b-2ee0cb54f0d9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:58.457814 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1e64e-4476-4aec-94f7-625167462316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:58.458454 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1e64e-4476-4aec-94f7-625167462316 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:58.458738 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1e64e-4476-4aec-94f7-625167462316 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:58.459014 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1e64e-4476-4aec-94f7-625167462316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:58.468486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1e64e-4476-4aec-94f7-625167462316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:58.470004 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1e64e-4476-4aec-94f7-625167462316 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:58.479830 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1401/2819] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:39:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:39:58.747318 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50d192-e0c7-47a5-81f3-6d5e6493b3e4 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:58.749522 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-aa50d192-e0c7-47a5-81f3-6d5e6493b3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:58.765679 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50d192-e0c7-47a5-81f3-6d5e6493b3e4 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:58.770430 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b3928-61bc-4430-9bdb-4a443733a8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:58.771203 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b3928-61bc-4430-9bdb-4a443733a8c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:58.771496 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b3928-61bc-4430-9bdb-4a443733a8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:58.771779 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b3928-61bc-4430-9bdb-4a443733a8c1 admin admin] PATH_INFO: `/v3/projects/661fd65930ca4599813ca34bf91e2b08` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:58.773121 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b3928-61bc-4430-9bdb-4a443733a8c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=661fd65930ca4599813ca34bf91e2b08)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:58.774994 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b3928-61bc-4430-9bdb-4a443733a8c1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:58.822802 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-282b3928-61bc-4430-9bdb-4a443733a8c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '661fd65930ca4599813ca34bf91e2b08', 'request_id': 'req-282b3928-61bc-4430-9bdb-4a443733a8c1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:58.828266 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-282b3928-61bc-4430-9bdb-4a443733a8c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '661fd65930ca4599813ca34bf91e2b08', 'request_id': 'req-282b3928-61bc-4430-9bdb-4a443733a8c1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:58.834392 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-282b3928-61bc-4430-9bdb-4a443733a8c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '661fd65930ca4599813ca34bf91e2b08', 'request_id': 'req-282b3928-61bc-4430-9bdb-4a443733a8c1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:58.847269 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-282b3928-61bc-4430-9bdb-4a443733a8c1 admin admin] The token cache is being invalidate because project 661fd65930ca4599813ca34bf91e2b08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:39:58.847552 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-282b3928-61bc-4430-9bdb-4a443733a8c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:58.848161 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-282b3928-61bc-4430-9bdb-4a443733a8c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:58.849724 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1419/2820] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:39:58 2022] DELETE /identity/v3/projects/661fd65930ca4599813ca34bf91e2b08 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:39:58.854001 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1e64e-4476-4aec-94f7-625167462316 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:58.864239 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1e64e-4476-4aec-94f7-625167462316 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:58.910207 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:58.922376 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ebe44-d223-4c2f-8506-26502e4f073b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:58.923322 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ebe44-d223-4c2f-8506-26502e4f073b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:58.923799 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ebe44-d223-4c2f-8506-26502e4f073b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:58.923967 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ebe44-d223-4c2f-8506-26502e4f073b admin admin] PATH_INFO: `/v3/users/18e0df3304254c36864cd3ed685ba635` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:58.925289 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ebe44-d223-4c2f-8506-26502e4f073b admin admin] RBAC: Authorizing `identity:delete_user(user_id=18e0df3304254c36864cd3ed685ba635)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:58.927404 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ebe44-d223-4c2f-8506-26502e4f073b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:58.975922 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-262ebe44-d223-4c2f-8506-26502e4f073b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18e0df3304254c36864cd3ed685ba635'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:58.976608 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-262ebe44-d223-4c2f-8506-26502e4f073b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18e0df3304254c36864cd3ed685ba635'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:58.977214 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-262ebe44-d223-4c2f-8506-26502e4f073b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18e0df3304254c36864cd3ed685ba635'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:58.984238 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-262ebe44-d223-4c2f-8506-26502e4f073b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18e0df3304254c36864cd3ed685ba635'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:58.996531 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-262ebe44-d223-4c2f-8506-26502e4f073b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18e0df3304254c36864cd3ed685ba635'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:59.010314 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1402/2821] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:39:58 2022] DELETE /identity/v3/users/18e0df3304254c36864cd3ed685ba635 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:39:59.290303 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b3928-61bc-4430-9bdb-4a443733a8c1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:59.291907 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-282b3928-61bc-4430-9bdb-4a443733a8c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:59.309365 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b3928-61bc-4430-9bdb-4a443733a8c1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:59.375880 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91792cf1-cef9-4a5e-a5be-f2d9b20fa01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:59.376898 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91792cf1-cef9-4a5e-a5be-f2d9b20fa01b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:59.377187 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91792cf1-cef9-4a5e-a5be-f2d9b20fa01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:59.377467 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91792cf1-cef9-4a5e-a5be-f2d9b20fa01b admin admin] PATH_INFO: `/v3/projects/245bc6b2a4524a4ebe0cab7d1c561fb0` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:59.378798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91792cf1-cef9-4a5e-a5be-f2d9b20fa01b admin admin] RBAC: Authorizing `identity:delete_project(project_id=245bc6b2a4524a4ebe0cab7d1c561fb0)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:59.381504 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91792cf1-cef9-4a5e-a5be-f2d9b20fa01b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:59.435952 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:59.444923 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-91792cf1-cef9-4a5e-a5be-f2d9b20fa01b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '245bc6b2a4524a4ebe0cab7d1c561fb0', 'request_id': 'req-91792cf1-cef9-4a5e-a5be-f2d9b20fa01b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:59.455592 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-91792cf1-cef9-4a5e-a5be-f2d9b20fa01b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '245bc6b2a4524a4ebe0cab7d1c561fb0', 'request_id': 'req-91792cf1-cef9-4a5e-a5be-f2d9b20fa01b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:59.459024 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-91792cf1-cef9-4a5e-a5be-f2d9b20fa01b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '245bc6b2a4524a4ebe0cab7d1c561fb0', 'request_id': 'req-91792cf1-cef9-4a5e-a5be-f2d9b20fa01b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:59.473459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-91792cf1-cef9-4a5e-a5be-f2d9b20fa01b admin admin] The token cache is being invalidate because project 245bc6b2a4524a4ebe0cab7d1c561fb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:39:59.473757 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-91792cf1-cef9-4a5e-a5be-f2d9b20fa01b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:59.474429 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-91792cf1-cef9-4a5e-a5be-f2d9b20fa01b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:39:59.476546 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1420/2822] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:39:59 2022] DELETE /identity/v3/projects/245bc6b2a4524a4ebe0cab7d1c561fb0 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:39:59.481662 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87740992-e6b8-4d54-b089-6180ca5d48e6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:59.482156 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87740992-e6b8-4d54-b089-6180ca5d48e6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:59.482429 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87740992-e6b8-4d54-b089-6180ca5d48e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:59.506433 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-87740992-e6b8-4d54-b089-6180ca5d48e6 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:59.507616 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-87740992-e6b8-4d54-b089-6180ca5d48e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:59.572203 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1403/2823] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:59.586090 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525b973-7f46-4a9f-8da5-215a788914e8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:59.586386 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525b973-7f46-4a9f-8da5-215a788914e8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:59.586671 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525b973-7f46-4a9f-8da5-215a788914e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:59.610403 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6525b973-7f46-4a9f-8da5-215a788914e8 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:59.611656 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6525b973-7f46-4a9f-8da5-215a788914e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:39:59.619853 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1421/2824] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:39:59.624744 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87740992-e6b8-4d54-b089-6180ca5d48e6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:59.634441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87740992-e6b8-4d54-b089-6180ca5d48e6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:59.639008 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd61fd-4445-4207-aa71-d2fa9b21d3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:59.640262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd61fd-4445-4207-aa71-d2fa9b21d3d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:59.640582 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd61fd-4445-4207-aa71-d2fa9b21d3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:59.640872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd61fd-4445-4207-aa71-d2fa9b21d3d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:59.641615 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd61fd-4445-4207-aa71-d2fa9b21d3d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:39:59.641926 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd61fd-4445-4207-aa71-d2fa9b21d3d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:59.643530 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd61fd-4445-4207-aa71-d2fa9b21d3d0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:59.656112 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1404/2825] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:39:59.660636 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525b973-7f46-4a9f-8da5-215a788914e8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:59.670558 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525b973-7f46-4a9f-8da5-215a788914e8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:59.675301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29e76d-43e1-47e1-ab37-7f810cc90580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:59.675967 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29e76d-43e1-47e1-ab37-7f810cc90580 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:59.676258 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29e76d-43e1-47e1-ab37-7f810cc90580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:59.676570 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29e76d-43e1-47e1-ab37-7f810cc90580 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:59.677290 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29e76d-43e1-47e1-ab37-7f810cc90580 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:59.679348 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29e76d-43e1-47e1-ab37-7f810cc90580 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:59.712581 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1422/2826] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:39:59 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:39:59.714000 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd61fd-4445-4207-aa71-d2fa9b21d3d0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:59.725086 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd61fd-4445-4207-aa71-d2fa9b21d3d0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:59.777772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:39:59.785511 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d859c4f-f1bf-45c8-9ac9-76dc23dcf85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:59.786185 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d859c4f-f1bf-45c8-9ac9-76dc23dcf85e admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:59.786487 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d859c4f-f1bf-45c8-9ac9-76dc23dcf85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:59.786977 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d859c4f-f1bf-45c8-9ac9-76dc23dcf85e admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:59.787760 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d859c4f-f1bf-45c8-9ac9-76dc23dcf85e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:59.790165 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d859c4f-f1bf-45c8-9ac9-76dc23dcf85e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:59.809562 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1405/2827] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:39:59 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:39:59.813717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29e76d-43e1-47e1-ab37-7f810cc90580 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:59.824193 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29e76d-43e1-47e1-ab37-7f810cc90580 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:59.828958 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7c60f-cb0f-40e6-a883-c30cd3d47ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:59.829560 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7c60f-cb0f-40e6-a883-c30cd3d47ebb admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:59.829868 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7c60f-cb0f-40e6-a883-c30cd3d47ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:59.830151 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7c60f-cb0f-40e6-a883-c30cd3d47ebb admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:59.830918 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7c60f-cb0f-40e6-a883-c30cd3d47ebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:59.832578 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7c60f-cb0f-40e6-a883-c30cd3d47ebb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:59.843440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1423/2828] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:39:59.847306 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d859c4f-f1bf-45c8-9ac9-76dc23dcf85e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:59.857473 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d859c4f-f1bf-45c8-9ac9-76dc23dcf85e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:59.862688 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1f519-b775-4132-8416-992a456338cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:59.863351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1f519-b775-4132-8416-992a456338cc admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:59.863634 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1f519-b775-4132-8416-992a456338cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:59.863913 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1f519-b775-4132-8416-992a456338cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:59.864679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1f519-b775-4132-8416-992a456338cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:59.866275 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1f519-b775-4132-8416-992a456338cc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:59.876233 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1406/2829] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:39:59.880162 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7c60f-cb0f-40e6-a883-c30cd3d47ebb admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:39:59.890818 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7c60f-cb0f-40e6-a883-c30cd3d47ebb admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:39:59.895396 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d1885-d46e-4fe2-bdeb-aad58b712bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:39:59.896283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d1885-d46e-4fe2-bdeb-aad58b712bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:59.896641 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d1885-d46e-4fe2-bdeb-aad58b712bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:59.896893 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d1885-d46e-4fe2-bdeb-aad58b712bfc admin admin] PATH_INFO: `/v3/projects/c745651bdf004cdab92b14920848e2b2/users/82edaa1595e44dc7a415ec86195cdc6b/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:59.911831 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d1885-d46e-4fe2-bdeb-aad58b712bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c745651bdf004cdab92b14920848e2b2, user_id=82edaa1595e44dc7a415ec86195cdc6b, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:39:59.914666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d1885-d46e-4fe2-bdeb-aad58b712bfc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:39:59.928154 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1424/2830] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:39:59 2022] PUT /identity/v3/projects/c745651bdf004cdab92b14920848e2b2/users/82edaa1595e44dc7a415ec86195cdc6b/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:39:59.939067 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e70734-6227-4856-b036-fdc3ad2ee6f9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:39:59.939366 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e70734-6227-4856-b036-fdc3ad2ee6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:39:59.939653 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e70734-6227-4856-b036-fdc3ad2ee6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:39:59.970600 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-34e70734-6227-4856-b036-fdc3ad2ee6f9 None None] MFA Rules not processed for user `82edaa1595e44dc7a415ec86195cdc6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:39:59.971838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-34e70734-6227-4856-b036-fdc3ad2ee6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:00.034329 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1407/2831] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:39:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:00.038747 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d1885-d46e-4fe2-bdeb-aad58b712bfc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:00.099814 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d1885-d46e-4fe2-bdeb-aad58b712bfc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:00.104730 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d95f9b-41a8-40a1-993b-b1f0a060a9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:00.105386 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d95f9b-41a8-40a1-993b-b1f0a060a9c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:00.105673 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d95f9b-41a8-40a1-993b-b1f0a060a9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:00.105953 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d95f9b-41a8-40a1-993b-b1f0a060a9c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:00.106654 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d95f9b-41a8-40a1-993b-b1f0a060a9c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:00.109176 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d95f9b-41a8-40a1-993b-b1f0a060a9c1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:00.134299 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1425/2832] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:40:00 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:40:00.138313 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e70734-6227-4856-b036-fdc3ad2ee6f9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:00.148907 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e70734-6227-4856-b036-fdc3ad2ee6f9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:00.204389 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14e910-b3e6-49e7-8f13-0532944bc51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:00.205087 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14e910-b3e6-49e7-8f13-0532944bc51d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:00.205374 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14e910-b3e6-49e7-8f13-0532944bc51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:00.205652 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14e910-b3e6-49e7-8f13-0532944bc51d admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:00.206355 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14e910-b3e6-49e7-8f13-0532944bc51d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:00.208277 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14e910-b3e6-49e7-8f13-0532944bc51d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:00.227406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1408/2833] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:40:00 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 10:40:00.231625 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d95f9b-41a8-40a1-993b-b1f0a060a9c1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:00.242644 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d95f9b-41a8-40a1-993b-b1f0a060a9c1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:00.247422 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d8ee0-07a2-4d7c-b849-a29597b15011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:00.248078 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d8ee0-07a2-4d7c-b849-a29597b15011 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:00.248362 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d8ee0-07a2-4d7c-b849-a29597b15011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:00.248674 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d8ee0-07a2-4d7c-b849-a29597b15011 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:00.249973 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d8ee0-07a2-4d7c-b849-a29597b15011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:00.251567 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d8ee0-07a2-4d7c-b849-a29597b15011 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:00.261990 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1426/2834] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:40:00.266537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14e910-b3e6-49e7-8f13-0532944bc51d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:00.276305 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14e910-b3e6-49e7-8f13-0532944bc51d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:00.281485 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0dee4-7146-4cc1-b10a-42efab914f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:00.282347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0dee4-7146-4cc1-b10a-42efab914f13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:00.282630 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0dee4-7146-4cc1-b10a-42efab914f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:00.282909 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0dee4-7146-4cc1-b10a-42efab914f13 admin admin] PATH_INFO: `/v3/projects/fb374419b2c346178673ae0d59047770/users/320b4f4298b14f64be01698a944c6124/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:00.297323 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e0dee4-7146-4cc1-b10a-42efab914f13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb374419b2c346178673ae0d59047770, user_id=320b4f4298b14f64be01698a944c6124, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:00.300118 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e0dee4-7146-4cc1-b10a-42efab914f13 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:00.313091 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1409/2835] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:40:00 2022] PUT /identity/v3/projects/fb374419b2c346178673ae0d59047770/users/320b4f4298b14f64be01698a944c6124/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:40:00.317021 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d8ee0-07a2-4d7c-b849-a29597b15011 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:00.327659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d8ee0-07a2-4d7c-b849-a29597b15011 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:00.364667 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:00.384956 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e40af2-6dab-45ff-a084-54e91eaba601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:00.385621 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e40af2-6dab-45ff-a084-54e91eaba601 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:00.385902 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e40af2-6dab-45ff-a084-54e91eaba601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:00.386179 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e40af2-6dab-45ff-a084-54e91eaba601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:00.386927 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e40af2-6dab-45ff-a084-54e91eaba601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:00.388543 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e40af2-6dab-45ff-a084-54e91eaba601 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:00.398723 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1427/2836] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:40:00.403373 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0dee4-7146-4cc1-b10a-42efab914f13 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:00.414233 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0dee4-7146-4cc1-b10a-42efab914f13 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:00.418833 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737c8ad-4775-4c6f-b031-497b10c3b4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:00.419713 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737c8ad-4775-4c6f-b031-497b10c3b4f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:00.419997 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737c8ad-4775-4c6f-b031-497b10c3b4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:00.420279 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737c8ad-4775-4c6f-b031-497b10c3b4f8 admin admin] PATH_INFO: `/v3/projects/fb374419b2c346178673ae0d59047770/users/320b4f4298b14f64be01698a944c6124/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:00.422872 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737c8ad-4775-4c6f-b031-497b10c3b4f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb374419b2c346178673ae0d59047770, user_id=320b4f4298b14f64be01698a944c6124, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:00.425596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737c8ad-4775-4c6f-b031-497b10c3b4f8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:00.436059 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1410/2837] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:40:00 2022] PUT /identity/v3/projects/fb374419b2c346178673ae0d59047770/users/320b4f4298b14f64be01698a944c6124/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:40:00.445986 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85668d02-a7ef-46b0-a5a8-8248ced0ae7c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:00.446307 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85668d02-a7ef-46b0-a5a8-8248ced0ae7c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:00.446594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85668d02-a7ef-46b0-a5a8-8248ced0ae7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:00.472847 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-85668d02-a7ef-46b0-a5a8-8248ced0ae7c None None] MFA Rules not processed for user `320b4f4298b14f64be01698a944c6124`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:00.474040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-85668d02-a7ef-46b0-a5a8-8248ced0ae7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:00.544329 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1428/2838] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:40:00 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:00.556083 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737c8ad-4775-4c6f-b031-497b10c3b4f8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:00.557924 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5737c8ad-4775-4c6f-b031-497b10c3b4f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:00.579372 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:00.619268 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737c8ad-4775-4c6f-b031-497b10c3b4f8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:00.622965 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f24094-61bf-4fee-a14d-bc8951519eee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:00.623628 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f24094-61bf-4fee-a14d-bc8951519eee service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:00.623917 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f24094-61bf-4fee-a14d-bc8951519eee service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:00.624200 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f24094-61bf-4fee-a14d-bc8951519eee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:00.633808 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f24094-61bf-4fee-a14d-bc8951519eee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:00.635274 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f24094-61bf-4fee-a14d-bc8951519eee service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:00.645097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1411/2839] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:40:00 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:40:00.845350 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85668d02-a7ef-46b0-a5a8-8248ced0ae7c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:00.846890 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-85668d02-a7ef-46b0-a5a8-8248ced0ae7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:00.909003 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85668d02-a7ef-46b0-a5a8-8248ced0ae7c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:00.912631 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07120c4c-a62b-40ca-ac1f-15f2b15dfca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:00.913283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07120c4c-a62b-40ca-ac1f-15f2b15dfca4 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:00.913573 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07120c4c-a62b-40ca-ac1f-15f2b15dfca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:00.913851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07120c4c-a62b-40ca-ac1f-15f2b15dfca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:00.915659 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-07120c4c-a62b-40ca-ac1f-15f2b15dfca4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:00.993292 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07120c4c-a62b-40ca-ac1f-15f2b15dfca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:00.994776 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07120c4c-a62b-40ca-ac1f-15f2b15dfca4 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:01.011165 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1429/2840] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:40:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:40:01.078820 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f24094-61bf-4fee-a14d-bc8951519eee service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:01.089859 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f24094-61bf-4fee-a14d-bc8951519eee service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:01.095172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1733d28-c91b-4276-ba71-5211a7c22f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:01.096063 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1733d28-c91b-4276-ba71-5211a7c22f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:01.096342 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1733d28-c91b-4276-ba71-5211a7c22f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:01.096650 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1733d28-c91b-4276-ba71-5211a7c22f92 admin admin] PATH_INFO: `/v3/users/82edaa1595e44dc7a415ec86195cdc6b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:01.097903 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1733d28-c91b-4276-ba71-5211a7c22f92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82edaa1595e44dc7a415ec86195cdc6b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:01.099846 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1733d28-c91b-4276-ba71-5211a7c22f92 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:01.149238 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e1733d28-c91b-4276-ba71-5211a7c22f92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82edaa1595e44dc7a415ec86195cdc6b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:01.149890 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e1733d28-c91b-4276-ba71-5211a7c22f92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82edaa1595e44dc7a415ec86195cdc6b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:01.150506 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e1733d28-c91b-4276-ba71-5211a7c22f92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82edaa1595e44dc7a415ec86195cdc6b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:01.162718 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e1733d28-c91b-4276-ba71-5211a7c22f92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82edaa1595e44dc7a415ec86195cdc6b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:01.180379 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e1733d28-c91b-4276-ba71-5211a7c22f92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82edaa1595e44dc7a415ec86195cdc6b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:01.195032 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1412/2841] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:40:01 2022] DELETE /identity/v3/users/82edaa1595e44dc7a415ec86195cdc6b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:40:01.199751 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c6b53-63f5-454c-9cc6-d97a654ca599 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:01.200050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c6b53-63f5-454c-9cc6-d97a654ca599 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:01.200389 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c6b53-63f5-454c-9cc6-d97a654ca599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:01.218506 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bd6c6b53-63f5-454c-9cc6-d97a654ca599 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:01.219737 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bd6c6b53-63f5-454c-9cc6-d97a654ca599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:01.225281 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:01.283666 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1430/2842] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:01.298363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1733d28-c91b-4276-ba71-5211a7c22f92 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:01.299830 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e1733d28-c91b-4276-ba71-5211a7c22f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:01.372382 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1733d28-c91b-4276-ba71-5211a7c22f92 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:01.377011 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285cc9b-0e98-4571-8840-1b38b8392d33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:01.377678 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285cc9b-0e98-4571-8840-1b38b8392d33 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:01.377971 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285cc9b-0e98-4571-8840-1b38b8392d33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:01.378247 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285cc9b-0e98-4571-8840-1b38b8392d33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:01.388350 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0285cc9b-0e98-4571-8840-1b38b8392d33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:01.390468 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0285cc9b-0e98-4571-8840-1b38b8392d33 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:01.403400 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1413/2843] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:40:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:40:01.814660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c6b53-63f5-454c-9cc6-d97a654ca599 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:01.816187 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bd6c6b53-63f5-454c-9cc6-d97a654ca599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:01.828968 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c6b53-63f5-454c-9cc6-d97a654ca599 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:01.833917 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc683a-df21-48e7-8592-9fc5fffd0eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:01.834880 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc683a-df21-48e7-8592-9fc5fffd0eee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:01.835168 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc683a-df21-48e7-8592-9fc5fffd0eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:01.835445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc683a-df21-48e7-8592-9fc5fffd0eee admin admin] PATH_INFO: `/v3/projects/c745651bdf004cdab92b14920848e2b2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:01.836692 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc683a-df21-48e7-8592-9fc5fffd0eee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c745651bdf004cdab92b14920848e2b2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:01.838691 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc683a-df21-48e7-8592-9fc5fffd0eee admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:01.888040 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b0bc683a-df21-48e7-8592-9fc5fffd0eee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c745651bdf004cdab92b14920848e2b2', 'request_id': 'req-b0bc683a-df21-48e7-8592-9fc5fffd0eee'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:01.894033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b0bc683a-df21-48e7-8592-9fc5fffd0eee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c745651bdf004cdab92b14920848e2b2', 'request_id': 'req-b0bc683a-df21-48e7-8592-9fc5fffd0eee'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:01.900433 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b0bc683a-df21-48e7-8592-9fc5fffd0eee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c745651bdf004cdab92b14920848e2b2', 'request_id': 'req-b0bc683a-df21-48e7-8592-9fc5fffd0eee'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:01.914047 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b0bc683a-df21-48e7-8592-9fc5fffd0eee admin admin] The token cache is being invalidate because project c745651bdf004cdab92b14920848e2b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:40:01.914344 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b0bc683a-df21-48e7-8592-9fc5fffd0eee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:01.915438 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b0bc683a-df21-48e7-8592-9fc5fffd0eee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:01.917123 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1431/2844] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:40:01 2022] DELETE /identity/v3/projects/c745651bdf004cdab92b14920848e2b2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:40:01.921787 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285cc9b-0e98-4571-8840-1b38b8392d33 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:01.931963 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285cc9b-0e98-4571-8840-1b38b8392d33 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:01.965767 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:01.992415 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d4674-3ab3-43b7-96b8-9a8fe8d09440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:01.993347 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d4674-3ab3-43b7-96b8-9a8fe8d09440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:01.993645 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d4674-3ab3-43b7-96b8-9a8fe8d09440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:01.994029 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d4674-3ab3-43b7-96b8-9a8fe8d09440 admin admin] PATH_INFO: `/v3/users/320b4f4298b14f64be01698a944c6124` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:01.995239 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d4674-3ab3-43b7-96b8-9a8fe8d09440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=320b4f4298b14f64be01698a944c6124)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:01.997250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d4674-3ab3-43b7-96b8-9a8fe8d09440 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:02.053781 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f27d4674-3ab3-43b7-96b8-9a8fe8d09440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '320b4f4298b14f64be01698a944c6124'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:02.054442 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f27d4674-3ab3-43b7-96b8-9a8fe8d09440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '320b4f4298b14f64be01698a944c6124'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:02.055047 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f27d4674-3ab3-43b7-96b8-9a8fe8d09440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '320b4f4298b14f64be01698a944c6124'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:02.062148 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f27d4674-3ab3-43b7-96b8-9a8fe8d09440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '320b4f4298b14f64be01698a944c6124'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:02.073874 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f27d4674-3ab3-43b7-96b8-9a8fe8d09440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '320b4f4298b14f64be01698a944c6124'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:02.088734 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1414/2845] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:40:01 2022] DELETE /identity/v3/users/320b4f4298b14f64be01698a944c6124 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:40:02.381616 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc683a-df21-48e7-8592-9fc5fffd0eee admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:02.385572 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b0bc683a-df21-48e7-8592-9fc5fffd0eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:02.413014 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc683a-df21-48e7-8592-9fc5fffd0eee admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:02.490050 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c86266-d74c-46a9-8fb1-0cef898d6060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:02.490934 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c86266-d74c-46a9-8fb1-0cef898d6060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:02.491254 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c86266-d74c-46a9-8fb1-0cef898d6060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:02.491565 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c86266-d74c-46a9-8fb1-0cef898d6060 admin admin] PATH_INFO: `/v3/projects/fb374419b2c346178673ae0d59047770` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:02.492801 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c86266-d74c-46a9-8fb1-0cef898d6060 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb374419b2c346178673ae0d59047770)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:02.494885 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c86266-d74c-46a9-8fb1-0cef898d6060 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:02.535501 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:02.548600 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-10c86266-d74c-46a9-8fb1-0cef898d6060 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb374419b2c346178673ae0d59047770', 'request_id': 'req-10c86266-d74c-46a9-8fb1-0cef898d6060'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:02.554347 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-10c86266-d74c-46a9-8fb1-0cef898d6060 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb374419b2c346178673ae0d59047770', 'request_id': 'req-10c86266-d74c-46a9-8fb1-0cef898d6060'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:02.560851 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-10c86266-d74c-46a9-8fb1-0cef898d6060 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb374419b2c346178673ae0d59047770', 'request_id': 'req-10c86266-d74c-46a9-8fb1-0cef898d6060'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:02.574774 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-10c86266-d74c-46a9-8fb1-0cef898d6060 admin admin] The token cache is being invalidate because project fb374419b2c346178673ae0d59047770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:40:02.575069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-10c86266-d74c-46a9-8fb1-0cef898d6060 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:02.575713 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-10c86266-d74c-46a9-8fb1-0cef898d6060 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:02.577459 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1432/2846] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:40:02 2022] DELETE /identity/v3/projects/fb374419b2c346178673ae0d59047770 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:40:02.583005 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c82dc-9c37-48c3-84d3-9ef9eeba6418 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:02.583381 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c82dc-9c37-48c3-84d3-9ef9eeba6418 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:02.583669 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c82dc-9c37-48c3-84d3-9ef9eeba6418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:02.602489 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9b1c82dc-9c37-48c3-84d3-9ef9eeba6418 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:02.603707 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9b1c82dc-9c37-48c3-84d3-9ef9eeba6418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:02.667749 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1415/2847] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:02.729242 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96e733-1710-4338-8052-abbdc52f0e7d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:02.729541 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96e733-1710-4338-8052-abbdc52f0e7d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:02.729826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96e733-1710-4338-8052-abbdc52f0e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:02.748210 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-dc96e733-1710-4338-8052-abbdc52f0e7d None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:02.749899 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dc96e733-1710-4338-8052-abbdc52f0e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:02.757985 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1433/2848] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:02.762255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c82dc-9c37-48c3-84d3-9ef9eeba6418 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:02.772971 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c82dc-9c37-48c3-84d3-9ef9eeba6418 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:02.777754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743f52d-d30d-4a9c-b969-f6e41efd0dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:02.778633 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743f52d-d30d-4a9c-b969-f6e41efd0dfd admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:02.778918 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743f52d-d30d-4a9c-b969-f6e41efd0dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:02.779195 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743f52d-d30d-4a9c-b969-f6e41efd0dfd admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:02.779947 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743f52d-d30d-4a9c-b969-f6e41efd0dfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:40:02.783469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743f52d-d30d-4a9c-b969-f6e41efd0dfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:02.783469 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743f52d-d30d-4a9c-b969-f6e41efd0dfd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:02.799051 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1416/2849] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:40:02.800473 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96e733-1710-4338-8052-abbdc52f0e7d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:02.810214 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96e733-1710-4338-8052-abbdc52f0e7d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:02.815223 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe219d5-cd97-452f-afba-9ae3ce5317e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:02.815993 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe219d5-cd97-452f-afba-9ae3ce5317e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:02.816382 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe219d5-cd97-452f-afba-9ae3ce5317e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:02.816783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe219d5-cd97-452f-afba-9ae3ce5317e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:02.817604 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe219d5-cd97-452f-afba-9ae3ce5317e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:02.819764 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe219d5-cd97-452f-afba-9ae3ce5317e6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:02.852133 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1434/2850] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:40:02 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:40:02.854206 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743f52d-d30d-4a9c-b969-f6e41efd0dfd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:02.867449 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743f52d-d30d-4a9c-b969-f6e41efd0dfd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:02.912839 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:02.933603 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7f673-d41a-4f68-a9d1-cd840aed2e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:02.934325 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7f673-d41a-4f68-a9d1-cd840aed2e38 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:02.934553 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7f673-d41a-4f68-a9d1-cd840aed2e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:02.934828 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7f673-d41a-4f68-a9d1-cd840aed2e38 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:02.935521 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7f673-d41a-4f68-a9d1-cd840aed2e38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:02.938115 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7f673-d41a-4f68-a9d1-cd840aed2e38 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:02.961193 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1417/2851] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:40:02 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:40:02.965490 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe219d5-cd97-452f-afba-9ae3ce5317e6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:02.978515 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe219d5-cd97-452f-afba-9ae3ce5317e6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:02.984194 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a15958-e802-4b55-83ff-8c373a0d1b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:02.985038 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a15958-e802-4b55-83ff-8c373a0d1b95 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:02.985799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a15958-e802-4b55-83ff-8c373a0d1b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:02.985799 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a15958-e802-4b55-83ff-8c373a0d1b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:02.986591 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a15958-e802-4b55-83ff-8c373a0d1b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:02.988325 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a15958-e802-4b55-83ff-8c373a0d1b95 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:03.001134 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1435/2852] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 10:40:03.005869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7f673-d41a-4f68-a9d1-cd840aed2e38 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:03.019013 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7f673-d41a-4f68-a9d1-cd840aed2e38 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:03.023870 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745727e7-3ecd-400e-a95b-302fe08daf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:03.024578 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745727e7-3ecd-400e-a95b-302fe08daf29 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:03.024867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745727e7-3ecd-400e-a95b-302fe08daf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:03.025148 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745727e7-3ecd-400e-a95b-302fe08daf29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:03.026030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745727e7-3ecd-400e-a95b-302fe08daf29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:03.028019 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745727e7-3ecd-400e-a95b-302fe08daf29 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:03.042380 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1418/2853] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:40:03.043748 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a15958-e802-4b55-83ff-8c373a0d1b95 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:03.055428 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a15958-e802-4b55-83ff-8c373a0d1b95 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:03.061001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb860fed-8e2f-4a59-bf02-0e0281185095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:03.062163 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb860fed-8e2f-4a59-bf02-0e0281185095 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:03.062545 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb860fed-8e2f-4a59-bf02-0e0281185095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:03.062985 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb860fed-8e2f-4a59-bf02-0e0281185095 admin admin] PATH_INFO: `/v3/projects/c2e99bbedebd4daca6a8511222a98d15/users/be22194c49b24dd89bd6d5e7b97418ad/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:03.080247 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb860fed-8e2f-4a59-bf02-0e0281185095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e99bbedebd4daca6a8511222a98d15, user_id=be22194c49b24dd89bd6d5e7b97418ad, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:03.082859 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb860fed-8e2f-4a59-bf02-0e0281185095 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:03.098440 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1436/2854] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:40:03 2022] PUT /identity/v3/projects/c2e99bbedebd4daca6a8511222a98d15/users/be22194c49b24dd89bd6d5e7b97418ad/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:40:03.109524 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f66ef-b527-4e7c-81d4-e8e7dfb4ff61 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:03.109903 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f66ef-b527-4e7c-81d4-e8e7dfb4ff61 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:03.110277 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f66ef-b527-4e7c-81d4-e8e7dfb4ff61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:03.141533 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-544f66ef-b527-4e7c-81d4-e8e7dfb4ff61 None None] MFA Rules not processed for user `be22194c49b24dd89bd6d5e7b97418ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:03.142789 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-544f66ef-b527-4e7c-81d4-e8e7dfb4ff61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:03.233772 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1419/2855] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:40:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:03.238358 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb860fed-8e2f-4a59-bf02-0e0281185095 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:03.313396 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb860fed-8e2f-4a59-bf02-0e0281185095 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:03.318540 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c77df-44e1-496f-8e1c-e59325057a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:03.319324 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c77df-44e1-496f-8e1c-e59325057a94 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:03.320069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c77df-44e1-496f-8e1c-e59325057a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:03.320069 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c77df-44e1-496f-8e1c-e59325057a94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:03.320900 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c77df-44e1-496f-8e1c-e59325057a94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:03.322958 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c77df-44e1-496f-8e1c-e59325057a94 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:03.356876 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1437/2856] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:40:03 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:40:03.360914 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f66ef-b527-4e7c-81d4-e8e7dfb4ff61 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:03.371667 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f66ef-b527-4e7c-81d4-e8e7dfb4ff61 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:03.431808 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f52cab-e277-4192-808c-eb6b0e571a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:03.432753 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f52cab-e277-4192-808c-eb6b0e571a56 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:03.433078 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f52cab-e277-4192-808c-eb6b0e571a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:03.433307 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f52cab-e277-4192-808c-eb6b0e571a56 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:03.434037 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f52cab-e277-4192-808c-eb6b0e571a56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:03.436213 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f52cab-e277-4192-808c-eb6b0e571a56 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:03.461480 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1420/2857] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:40:03 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:40:03.462802 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c77df-44e1-496f-8e1c-e59325057a94 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:03.473021 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c77df-44e1-496f-8e1c-e59325057a94 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:03.477638 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ac8c2-b050-4cd0-a134-d46c972f72fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:03.478294 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ac8c2-b050-4cd0-a134-d46c972f72fb admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:03.478580 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ac8c2-b050-4cd0-a134-d46c972f72fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:03.479124 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ac8c2-b050-4cd0-a134-d46c972f72fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:03.479852 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ac8c2-b050-4cd0-a134-d46c972f72fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:03.481500 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ac8c2-b050-4cd0-a134-d46c972f72fb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:03.492091 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1438/2858] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:40:03.496231 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f52cab-e277-4192-808c-eb6b0e571a56 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:03.507461 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f52cab-e277-4192-808c-eb6b0e571a56 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:03.512265 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ade7f-45be-4c61-9101-2c32cf12c2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:03.513262 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ade7f-45be-4c61-9101-2c32cf12c2f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:03.513609 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ade7f-45be-4c61-9101-2c32cf12c2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:03.513946 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ade7f-45be-4c61-9101-2c32cf12c2f3 admin admin] PATH_INFO: `/v3/projects/cc94a7703b71491f99e2fd52369b0ae5/users/e46732c494b64811b5043b5ac096ac0e/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:03.530098 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ade7f-45be-4c61-9101-2c32cf12c2f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc94a7703b71491f99e2fd52369b0ae5, user_id=e46732c494b64811b5043b5ac096ac0e, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:03.532867 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ade7f-45be-4c61-9101-2c32cf12c2f3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:03.546176 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1421/2859] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:40:03 2022] PUT /identity/v3/projects/cc94a7703b71491f99e2fd52369b0ae5/users/e46732c494b64811b5043b5ac096ac0e/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:40:03.550167 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ac8c2-b050-4cd0-a134-d46c972f72fb admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:03.560143 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ac8c2-b050-4cd0-a134-d46c972f72fb admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:03.595583 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:03.635540 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff988e-6ebd-46ce-8442-7c5e923b3699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:03.636370 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff988e-6ebd-46ce-8442-7c5e923b3699 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:03.636751 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff988e-6ebd-46ce-8442-7c5e923b3699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:03.637100 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff988e-6ebd-46ce-8442-7c5e923b3699 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:03.637907 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ff988e-6ebd-46ce-8442-7c5e923b3699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:03.640430 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ff988e-6ebd-46ce-8442-7c5e923b3699 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:03.652173 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1439/2860] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:40:03.660810 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ade7f-45be-4c61-9101-2c32cf12c2f3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:03.675202 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ade7f-45be-4c61-9101-2c32cf12c2f3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:03.680668 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550899ba-af40-41dc-88a0-188c7d9091ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:03.681661 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550899ba-af40-41dc-88a0-188c7d9091ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:03.682012 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550899ba-af40-41dc-88a0-188c7d9091ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:03.682354 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550899ba-af40-41dc-88a0-188c7d9091ed admin admin] PATH_INFO: `/v3/projects/cc94a7703b71491f99e2fd52369b0ae5/users/e46732c494b64811b5043b5ac096ac0e/roles/4d85c69b727b4fac8d2b15e4446bc949` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:03.688604 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550899ba-af40-41dc-88a0-188c7d9091ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc94a7703b71491f99e2fd52369b0ae5, user_id=e46732c494b64811b5043b5ac096ac0e, role_id=4d85c69b727b4fac8d2b15e4446bc949)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:03.691933 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550899ba-af40-41dc-88a0-188c7d9091ed admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:03.715349 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1422/2861] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:40:03 2022] PUT /identity/v3/projects/cc94a7703b71491f99e2fd52369b0ae5/users/e46732c494b64811b5043b5ac096ac0e/roles/4d85c69b727b4fac8d2b15e4446bc949 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:40:03.716454 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af777f76-a089-4de5-8443-3a3d56383d6a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:03.716785 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af777f76-a089-4de5-8443-3a3d56383d6a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:03.717057 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af777f76-a089-4de5-8443-3a3d56383d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:03.879915 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-af777f76-a089-4de5-8443-3a3d56383d6a None None] MFA Rules not processed for user `e46732c494b64811b5043b5ac096ac0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:03.881180 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-af777f76-a089-4de5-8443-3a3d56383d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:03.957585 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1440/2862] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:40:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:03.966015 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550899ba-af40-41dc-88a0-188c7d9091ed admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:03.968609 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-550899ba-af40-41dc-88a0-188c7d9091ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:03.974649 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:04.038915 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550899ba-af40-41dc-88a0-188c7d9091ed admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:04.042664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1f583-ad02-41e4-b6a0-c0d198c5f971 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:04.043250 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1f583-ad02-41e4-b6a0-c0d198c5f971 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:04.043535 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1f583-ad02-41e4-b6a0-c0d198c5f971 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:04.043812 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1f583-ad02-41e4-b6a0-c0d198c5f971 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:04.131198 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1f583-ad02-41e4-b6a0-c0d198c5f971 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:04.132768 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1f583-ad02-41e4-b6a0-c0d198c5f971 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:04.144302 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1423/2863] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:40:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:40:07.789301 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af777f76-a089-4de5-8443-3a3d56383d6a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:07.791768 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-af777f76-a089-4de5-8443-3a3d56383d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:07.858624 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af777f76-a089-4de5-8443-3a3d56383d6a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:07.863096 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6435ade-a179-493c-a9d3-535055d6e04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:07.863959 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6435ade-a179-493c-a9d3-535055d6e04d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:07.864240 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6435ade-a179-493c-a9d3-535055d6e04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:07.864530 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6435ade-a179-493c-a9d3-535055d6e04d admin admin] PATH_INFO: `/v3/users/4e9eb8d188e040bca894e038955f2402` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:07.865711 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6435ade-a179-493c-a9d3-535055d6e04d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e9eb8d188e040bca894e038955f2402)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:07.867620 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6435ade-a179-493c-a9d3-535055d6e04d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:07.913594 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e6435ade-a179-493c-a9d3-535055d6e04d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e9eb8d188e040bca894e038955f2402'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:07.921655 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:07.927632 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e6435ade-a179-493c-a9d3-535055d6e04d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e9eb8d188e040bca894e038955f2402'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:07.928629 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e6435ade-a179-493c-a9d3-535055d6e04d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e9eb8d188e040bca894e038955f2402'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:07.929413 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e6435ade-a179-493c-a9d3-535055d6e04d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e9eb8d188e040bca894e038955f2402'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:07.937030 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e6435ade-a179-493c-a9d3-535055d6e04d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e9eb8d188e040bca894e038955f2402'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:07.949028 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1441/2864] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:40:07 2022] DELETE /identity/v3/users/4e9eb8d188e040bca894e038955f2402 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:40:08.120220 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1f583-ad02-41e4-b6a0-c0d198c5f971 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:08.131484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1f583-ad02-41e4-b6a0-c0d198c5f971 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:08.191182 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b57d9-c64e-4ca7-8373-4189bd57bbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:08.192042 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b57d9-c64e-4ca7-8373-4189bd57bbfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:08.192330 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b57d9-c64e-4ca7-8373-4189bd57bbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:08.192640 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b57d9-c64e-4ca7-8373-4189bd57bbfd admin admin] PATH_INFO: `/v3/projects/2c656b71bbd447cba7d11335c928dfda` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:08.193832 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b57d9-c64e-4ca7-8373-4189bd57bbfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c656b71bbd447cba7d11335c928dfda)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:08.195755 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b57d9-c64e-4ca7-8373-4189bd57bbfd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:08.242020 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-476b57d9-c64e-4ca7-8373-4189bd57bbfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c656b71bbd447cba7d11335c928dfda', 'request_id': 'req-476b57d9-c64e-4ca7-8373-4189bd57bbfd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:08.247417 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-476b57d9-c64e-4ca7-8373-4189bd57bbfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c656b71bbd447cba7d11335c928dfda', 'request_id': 'req-476b57d9-c64e-4ca7-8373-4189bd57bbfd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:08.248633 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:08.255434 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-476b57d9-c64e-4ca7-8373-4189bd57bbfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c656b71bbd447cba7d11335c928dfda', 'request_id': 'req-476b57d9-c64e-4ca7-8373-4189bd57bbfd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:08.268111 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-476b57d9-c64e-4ca7-8373-4189bd57bbfd admin admin] The token cache is being invalidate because project 2c656b71bbd447cba7d11335c928dfda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:40:08.268406 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-476b57d9-c64e-4ca7-8373-4189bd57bbfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:08.269070 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-476b57d9-c64e-4ca7-8373-4189bd57bbfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:08.270606 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1424/2865] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:40:08 2022] DELETE /identity/v3/projects/2c656b71bbd447cba7d11335c928dfda => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:40:08.276783 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6435ade-a179-493c-a9d3-535055d6e04d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:08.279315 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e6435ade-a179-493c-a9d3-535055d6e04d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:08.293696 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6435ade-a179-493c-a9d3-535055d6e04d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:08.348401 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c969e23-809f-4b48-a325-89c4b5233225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:08.349302 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c969e23-809f-4b48-a325-89c4b5233225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:08.349589 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c969e23-809f-4b48-a325-89c4b5233225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:08.349868 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c969e23-809f-4b48-a325-89c4b5233225 admin admin] PATH_INFO: `/v3/users/cd1c16a2086446c7b8cacd4a81f9af71` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:08.351052 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c969e23-809f-4b48-a325-89c4b5233225 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd1c16a2086446c7b8cacd4a81f9af71)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:08.353016 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c969e23-809f-4b48-a325-89c4b5233225 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:08.398342 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1c969e23-809f-4b48-a325-89c4b5233225 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd1c16a2086446c7b8cacd4a81f9af71'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:08.410047 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1c969e23-809f-4b48-a325-89c4b5233225 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd1c16a2086446c7b8cacd4a81f9af71'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:08.410714 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1c969e23-809f-4b48-a325-89c4b5233225 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd1c16a2086446c7b8cacd4a81f9af71'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:08.411323 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1c969e23-809f-4b48-a325-89c4b5233225 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd1c16a2086446c7b8cacd4a81f9af71'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:08.418197 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1c969e23-809f-4b48-a325-89c4b5233225 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd1c16a2086446c7b8cacd4a81f9af71'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:08.430798 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1442/2866] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:40:08 2022] DELETE /identity/v3/users/cd1c16a2086446c7b8cacd4a81f9af71 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:40:08.600990 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b57d9-c64e-4ca7-8373-4189bd57bbfd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:08.602537 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-476b57d9-c64e-4ca7-8373-4189bd57bbfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:08.616805 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b57d9-c64e-4ca7-8373-4189bd57bbfd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:08.681558 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092a853-3d6f-454c-9b8c-90446bb80ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:08.682651 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092a853-3d6f-454c-9b8c-90446bb80ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:08.683015 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092a853-3d6f-454c-9b8c-90446bb80ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:08.683361 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092a853-3d6f-454c-9b8c-90446bb80ea1 admin admin] PATH_INFO: `/v3/projects/e6b6505e83184b2f8d54ad2ea4a1b9c4` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:08.684664 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092a853-3d6f-454c-9b8c-90446bb80ea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6b6505e83184b2f8d54ad2ea4a1b9c4)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:08.687255 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092a853-3d6f-454c-9b8c-90446bb80ea1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:08.742502 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b092a853-3d6f-454c-9b8c-90446bb80ea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6b6505e83184b2f8d54ad2ea4a1b9c4', 'request_id': 'req-b092a853-3d6f-454c-9b8c-90446bb80ea1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:08.745847 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c969e23-809f-4b48-a325-89c4b5233225 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:08.748001 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1c969e23-809f-4b48-a325-89c4b5233225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:08.748378 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b092a853-3d6f-454c-9b8c-90446bb80ea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6b6505e83184b2f8d54ad2ea4a1b9c4', 'request_id': 'req-b092a853-3d6f-454c-9b8c-90446bb80ea1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:08.755729 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b092a853-3d6f-454c-9b8c-90446bb80ea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6b6505e83184b2f8d54ad2ea4a1b9c4', 'request_id': 'req-b092a853-3d6f-454c-9b8c-90446bb80ea1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:08.769732 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b092a853-3d6f-454c-9b8c-90446bb80ea1 admin admin] The token cache is being invalidate because project e6b6505e83184b2f8d54ad2ea4a1b9c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:40:08.770031 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b092a853-3d6f-454c-9b8c-90446bb80ea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:08.770659 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b092a853-3d6f-454c-9b8c-90446bb80ea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:08.772409 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1425/2867] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:40:08 2022] DELETE /identity/v3/projects/e6b6505e83184b2f8d54ad2ea4a1b9c4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:40:08.814668 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c969e23-809f-4b48-a325-89c4b5233225 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:08.823888 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:08.870275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0121b-4fa7-4395-949b-c213a93cea6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:08.870942 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0121b-4fa7-4395-949b-c213a93cea6e service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:08.871224 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0121b-4fa7-4395-949b-c213a93cea6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:08.871494 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0121b-4fa7-4395-949b-c213a93cea6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:08.873363 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f6c0121b-4fa7-4395-949b-c213a93cea6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:08.942385 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c0121b-4fa7-4395-949b-c213a93cea6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:08.943750 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c0121b-4fa7-4395-949b-c213a93cea6e service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:08.954637 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1443/2868] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:40:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 10:40:13.803467 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092a853-3d6f-454c-9b8c-90446bb80ea1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:13.804955 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b092a853-3d6f-454c-9b8c-90446bb80ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:13.868846 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092a853-3d6f-454c-9b8c-90446bb80ea1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:13.873441 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d43f2b-460d-4597-8359-5c36cd5e072a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:13.874302 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d43f2b-460d-4597-8359-5c36cd5e072a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:13.874589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d43f2b-460d-4597-8359-5c36cd5e072a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:13.874869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d43f2b-460d-4597-8359-5c36cd5e072a admin admin] PATH_INFO: `/v3/users/be22194c49b24dd89bd6d5e7b97418ad` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:13.876064 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d43f2b-460d-4597-8359-5c36cd5e072a admin admin] RBAC: Authorizing `identity:delete_user(user_id=be22194c49b24dd89bd6d5e7b97418ad)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:13.878030 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d43f2b-460d-4597-8359-5c36cd5e072a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:13.898486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:13.925693 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21d43f2b-460d-4597-8359-5c36cd5e072a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be22194c49b24dd89bd6d5e7b97418ad'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:13.926429 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21d43f2b-460d-4597-8359-5c36cd5e072a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be22194c49b24dd89bd6d5e7b97418ad'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:13.927066 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21d43f2b-460d-4597-8359-5c36cd5e072a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be22194c49b24dd89bd6d5e7b97418ad'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:13.933966 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21d43f2b-460d-4597-8359-5c36cd5e072a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be22194c49b24dd89bd6d5e7b97418ad'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:13.946400 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-21d43f2b-460d-4597-8359-5c36cd5e072a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be22194c49b24dd89bd6d5e7b97418ad'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:13.958802 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1426/2869] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:40:13 2022] DELETE /identity/v3/users/be22194c49b24dd89bd6d5e7b97418ad => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:40:13.963448 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69ce14-3f38-4582-aa36-cf706a21df57 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:13.963930 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69ce14-3f38-4582-aa36-cf706a21df57 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:13.964033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69ce14-3f38-4582-aa36-cf706a21df57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:13.980740 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1e69ce14-3f38-4582-aa36-cf706a21df57 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:13.981924 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1e69ce14-3f38-4582-aa36-cf706a21df57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:14.040283 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1444/2870] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:14.050120 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d43f2b-460d-4597-8359-5c36cd5e072a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:14.052097 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-21d43f2b-460d-4597-8359-5c36cd5e072a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:14.112951 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d43f2b-460d-4597-8359-5c36cd5e072a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:14.116638 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252da71a-c4ec-4ca1-b038-6ba07bbdb205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:14.117308 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252da71a-c4ec-4ca1-b038-6ba07bbdb205 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:14.117596 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252da71a-c4ec-4ca1-b038-6ba07bbdb205 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:14.117875 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252da71a-c4ec-4ca1-b038-6ba07bbdb205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:14.127613 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252da71a-c4ec-4ca1-b038-6ba07bbdb205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:14.129090 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252da71a-c4ec-4ca1-b038-6ba07bbdb205 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:14.138519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1427/2871] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:40:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:40:14.401894 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69ce14-3f38-4582-aa36-cf706a21df57 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:14.403306 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1e69ce14-3f38-4582-aa36-cf706a21df57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:14.415126 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69ce14-3f38-4582-aa36-cf706a21df57 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:14.419918 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1390f-449e-445d-8554-f41891086764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:14.421387 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1390f-449e-445d-8554-f41891086764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:14.421675 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1390f-449e-445d-8554-f41891086764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:14.421950 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1390f-449e-445d-8554-f41891086764 admin admin] PATH_INFO: `/v3/projects/c2e99bbedebd4daca6a8511222a98d15` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:14.423132 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1390f-449e-445d-8554-f41891086764 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e99bbedebd4daca6a8511222a98d15)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:14.425115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1390f-449e-445d-8554-f41891086764 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:14.471919 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3cb1390f-449e-445d-8554-f41891086764 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e99bbedebd4daca6a8511222a98d15', 'request_id': 'req-3cb1390f-449e-445d-8554-f41891086764'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:14.476464 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:14.477660 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3cb1390f-449e-445d-8554-f41891086764 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e99bbedebd4daca6a8511222a98d15', 'request_id': 'req-3cb1390f-449e-445d-8554-f41891086764'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:14.485665 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3cb1390f-449e-445d-8554-f41891086764 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e99bbedebd4daca6a8511222a98d15', 'request_id': 'req-3cb1390f-449e-445d-8554-f41891086764'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:14.497608 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3cb1390f-449e-445d-8554-f41891086764 admin admin] The token cache is being invalidate because project c2e99bbedebd4daca6a8511222a98d15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:40:14.497895 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3cb1390f-449e-445d-8554-f41891086764 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:14.498527 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3cb1390f-449e-445d-8554-f41891086764 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:14.500739 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1445/2872] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:40:14 2022] DELETE /identity/v3/projects/c2e99bbedebd4daca6a8511222a98d15 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:40:14.504748 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252da71a-c4ec-4ca1-b038-6ba07bbdb205 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:14.514351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252da71a-c4ec-4ca1-b038-6ba07bbdb205 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:14.577679 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7786d815-faa2-4c24-92c6-ddb52b3cee42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:14.578472 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786d815-faa2-4c24-92c6-ddb52b3cee42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:14.578757 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786d815-faa2-4c24-92c6-ddb52b3cee42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:14.579034 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786d815-faa2-4c24-92c6-ddb52b3cee42 admin admin] PATH_INFO: `/v3/users/e46732c494b64811b5043b5ac096ac0e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:14.580210 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786d815-faa2-4c24-92c6-ddb52b3cee42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e46732c494b64811b5043b5ac096ac0e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:14.582165 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786d815-faa2-4c24-92c6-ddb52b3cee42 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:14.622720 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:14.628484 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7786d815-faa2-4c24-92c6-ddb52b3cee42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e46732c494b64811b5043b5ac096ac0e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:14.629519 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7786d815-faa2-4c24-92c6-ddb52b3cee42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e46732c494b64811b5043b5ac096ac0e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:14.630480 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7786d815-faa2-4c24-92c6-ddb52b3cee42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e46732c494b64811b5043b5ac096ac0e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:14.638008 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7786d815-faa2-4c24-92c6-ddb52b3cee42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e46732c494b64811b5043b5ac096ac0e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:14.649687 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7786d815-faa2-4c24-92c6-ddb52b3cee42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e46732c494b64811b5043b5ac096ac0e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:14.662361 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1428/2873] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:40:14 2022] DELETE /identity/v3/users/e46732c494b64811b5043b5ac096ac0e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:40:14.936180 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1390f-449e-445d-8554-f41891086764 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:14.937717 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3cb1390f-449e-445d-8554-f41891086764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:14.950850 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1390f-449e-445d-8554-f41891086764 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:15.012782 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447caee-e42a-417b-aa0c-5ed847128bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:15.013648 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447caee-e42a-417b-aa0c-5ed847128bb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:15.013932 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447caee-e42a-417b-aa0c-5ed847128bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:15.014209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447caee-e42a-417b-aa0c-5ed847128bb7 admin admin] PATH_INFO: `/v3/projects/cc94a7703b71491f99e2fd52369b0ae5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:15.015974 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447caee-e42a-417b-aa0c-5ed847128bb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc94a7703b71491f99e2fd52369b0ae5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:15.018062 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447caee-e42a-417b-aa0c-5ed847128bb7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:15.062524 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a447caee-e42a-417b-aa0c-5ed847128bb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc94a7703b71491f99e2fd52369b0ae5', 'request_id': 'req-a447caee-e42a-417b-aa0c-5ed847128bb7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:15.068011 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a447caee-e42a-417b-aa0c-5ed847128bb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc94a7703b71491f99e2fd52369b0ae5', 'request_id': 'req-a447caee-e42a-417b-aa0c-5ed847128bb7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:15.073983 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a447caee-e42a-417b-aa0c-5ed847128bb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc94a7703b71491f99e2fd52369b0ae5', 'request_id': 'req-a447caee-e42a-417b-aa0c-5ed847128bb7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:15.087808 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a447caee-e42a-417b-aa0c-5ed847128bb7 admin admin] The token cache is being invalidate because project cc94a7703b71491f99e2fd52369b0ae5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:40:15.087998 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a447caee-e42a-417b-aa0c-5ed847128bb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:15.088642 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a447caee-e42a-417b-aa0c-5ed847128bb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:15.090172 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1446/2874] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:40:14 2022] DELETE /identity/v3/projects/cc94a7703b71491f99e2fd52369b0ae5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:40:15.096917 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b12122-bded-4e8c-946d-5653824a611a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:15.097410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b12122-bded-4e8c-946d-5653824a611a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:15.097513 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b12122-bded-4e8c-946d-5653824a611a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:15.116147 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d7b12122-bded-4e8c-946d-5653824a611a None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:15.117375 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d7b12122-bded-4e8c-946d-5653824a611a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:15.176063 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1429/2875] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:15.184379 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98c23e-dd39-4de8-bb98-ee31e32bc872 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:15.184728 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98c23e-dd39-4de8-bb98-ee31e32bc872 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:15.185009 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98c23e-dd39-4de8-bb98-ee31e32bc872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:15.203547 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ac98c23e-dd39-4de8-bb98-ee31e32bc872 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:15.204775 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ac98c23e-dd39-4de8-bb98-ee31e32bc872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:15.212596 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1447/2876] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:15.216634 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b12122-bded-4e8c-946d-5653824a611a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:15.227233 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b12122-bded-4e8c-946d-5653824a611a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:15.232096 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d15bf-3da0-4371-b519-b4c9e673bcc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:15.232892 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d15bf-3da0-4371-b519-b4c9e673bcc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:15.233174 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d15bf-3da0-4371-b519-b4c9e673bcc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:15.233465 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d15bf-3da0-4371-b519-b4c9e673bcc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:15.234183 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d15bf-3da0-4371-b519-b4c9e673bcc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 10:40:15.234529 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d15bf-3da0-4371-b519-b4c9e673bcc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:15.236182 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d15bf-3da0-4371-b519-b4c9e673bcc6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:15.248194 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1430/2877] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jul 18 10:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 10:40:15.252586 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98c23e-dd39-4de8-bb98-ee31e32bc872 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:15.261780 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98c23e-dd39-4de8-bb98-ee31e32bc872 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:15.266901 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363275d-6254-44a5-a47f-2cf842dd94d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:15.267540 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363275d-6254-44a5-a47f-2cf842dd94d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:15.267826 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363275d-6254-44a5-a47f-2cf842dd94d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:15.268102 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363275d-6254-44a5-a47f-2cf842dd94d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:15.268828 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4363275d-6254-44a5-a47f-2cf842dd94d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:15.270726 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4363275d-6254-44a5-a47f-2cf842dd94d5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:15.295877 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1448/2878] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jul 18 10:40:15 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:40:15.300401 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d15bf-3da0-4371-b519-b4c9e673bcc6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:15.310226 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d15bf-3da0-4371-b519-b4c9e673bcc6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:15.365957 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b92bb-e16c-491e-9c61-a20ea8eb98c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:15.366617 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b92bb-e16c-491e-9c61-a20ea8eb98c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:15.366899 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b92bb-e16c-491e-9c61-a20ea8eb98c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:15.367173 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b92bb-e16c-491e-9c61-a20ea8eb98c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:15.367850 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b92bb-e16c-491e-9c61-a20ea8eb98c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:15.369774 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b92bb-e16c-491e-9c61-a20ea8eb98c4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:15.389144 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1431/2879] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jul 18 10:40:15 2022] POST /identity/v3/users => generated 292 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 10:40:15.393747 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363275d-6254-44a5-a47f-2cf842dd94d5 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:15.404473 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363275d-6254-44a5-a47f-2cf842dd94d5 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:15.412963 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2a501-dd28-45a8-832d-74ee098997b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:15.413767 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2a501-dd28-45a8-832d-74ee098997b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:15.414084 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2a501-dd28-45a8-832d-74ee098997b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:15.414361 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2a501-dd28-45a8-832d-74ee098997b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:15.415093 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2a501-dd28-45a8-832d-74ee098997b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:15.416781 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2a501-dd28-45a8-832d-74ee098997b1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:15.427913 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1449/2880] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:40:15.432058 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b92bb-e16c-491e-9c61-a20ea8eb98c4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:15.442595 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b92bb-e16c-491e-9c61-a20ea8eb98c4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:15.447204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551c79d-d146-411d-a84b-658b09745a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:15.447858 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551c79d-d146-411d-a84b-658b09745a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:15.448140 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551c79d-d146-411d-a84b-658b09745a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:15.448416 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551c79d-d146-411d-a84b-658b09745a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:15.449172 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3551c79d-d146-411d-a84b-658b09745a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:15.450754 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3551c79d-d146-411d-a84b-658b09745a59 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:15.461109 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1432/2881] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jul 18 10:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 10:40:15.465115 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2a501-dd28-45a8-832d-74ee098997b1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:15.475100 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2a501-dd28-45a8-832d-74ee098997b1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:15.479962 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98888384-6d6a-451c-9f2c-84d26fbf019d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:15.480873 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98888384-6d6a-451c-9f2c-84d26fbf019d admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:15.481158 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98888384-6d6a-451c-9f2c-84d26fbf019d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:15.481433 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98888384-6d6a-451c-9f2c-84d26fbf019d admin admin] PATH_INFO: `/v3/projects/80be84e153cf4739a12c043a4062ad5f/users/23f2106429664d5ea3d1b89083d199b5/roles/2820090979cb487db491f835e03e6471` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:15.495524 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98888384-6d6a-451c-9f2c-84d26fbf019d admin admin] RBAC: Authorizing `identity:create_grant(project_id=80be84e153cf4739a12c043a4062ad5f, user_id=23f2106429664d5ea3d1b89083d199b5, role_id=2820090979cb487db491f835e03e6471)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:15.498209 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98888384-6d6a-451c-9f2c-84d26fbf019d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:15.511233 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1450/2882] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jul 18 10:40:15 2022] PUT /identity/v3/projects/80be84e153cf4739a12c043a4062ad5f/users/23f2106429664d5ea3d1b89083d199b5/roles/2820090979cb487db491f835e03e6471 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:40:15.523263 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10af114-96fe-45d5-adaa-091a36a307bc None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:15.523583 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10af114-96fe-45d5-adaa-091a36a307bc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:15.523849 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10af114-96fe-45d5-adaa-091a36a307bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:15.551812 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c10af114-96fe-45d5-adaa-091a36a307bc None None] MFA Rules not processed for user `23f2106429664d5ea3d1b89083d199b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:15.553060 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c10af114-96fe-45d5-adaa-091a36a307bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:15.565230 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:15.618543 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1433/2883] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:40:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 10:40:15.629772 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98888384-6d6a-451c-9f2c-84d26fbf019d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:15.631413 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-98888384-6d6a-451c-9f2c-84d26fbf019d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:15.644951 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:15.689650 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98888384-6d6a-451c-9f2c-84d26fbf019d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:15.693302 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c395ffb-b0a7-48ce-82d1-e45eda5d1f6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e29c37dc924514a882f936f5936ae1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:15.693972 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c395ffb-b0a7-48ce-82d1-e45eda5d1f6d service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:15.694254 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c395ffb-b0a7-48ce-82d1-e45eda5d1f6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:15.694531 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c395ffb-b0a7-48ce-82d1-e45eda5d1f6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:15.703341 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c395ffb-b0a7-48ce-82d1-e45eda5d1f6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:15.704784 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c395ffb-b0a7-48ce-82d1-e45eda5d1f6d service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:15.713606 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1451/2884] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:40:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:40:23.022830 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10af114-96fe-45d5-adaa-091a36a307bc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:23.097193 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10af114-96fe-45d5-adaa-091a36a307bc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:23.102730 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800898f-4800-4818-b82d-4e97877c09e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:23.107486 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800898f-4800-4818-b82d-4e97877c09e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:23.107779 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800898f-4800-4818-b82d-4e97877c09e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:23.108065 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800898f-4800-4818-b82d-4e97877c09e4 admin admin] PATH_INFO: `/v3/users/23f2106429664d5ea3d1b89083d199b5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:23.109363 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800898f-4800-4818-b82d-4e97877c09e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23f2106429664d5ea3d1b89083d199b5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:23.111476 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800898f-4800-4818-b82d-4e97877c09e4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:23.165470 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a800898f-4800-4818-b82d-4e97877c09e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23f2106429664d5ea3d1b89083d199b5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:23.166126 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a800898f-4800-4818-b82d-4e97877c09e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23f2106429664d5ea3d1b89083d199b5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:23.166719 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a800898f-4800-4818-b82d-4e97877c09e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23f2106429664d5ea3d1b89083d199b5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:23.175058 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a800898f-4800-4818-b82d-4e97877c09e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23f2106429664d5ea3d1b89083d199b5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:23.187418 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a800898f-4800-4818-b82d-4e97877c09e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23f2106429664d5ea3d1b89083d199b5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:23.200838 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1434/2885] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jul 18 10:40:23 2022] DELETE /identity/v3/users/23f2106429664d5ea3d1b89083d199b5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 10:40:23.205445 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748df03-73a4-4ce0-8058-b134909e4e21 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:23.206033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748df03-73a4-4ce0-8058-b134909e4e21 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:23.206033 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748df03-73a4-4ce0-8058-b134909e4e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:23.222830 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3748df03-73a4-4ce0-8058-b134909e4e21 None None] MFA Rules not processed for user `06ec5a62dde8447fa4c9a00a0f1828f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 10:40:23.224191 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3748df03-73a4-4ce0-8058-b134909e4e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:23.286275 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1452/2886] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jul 18 10:40:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 10:40:23.296027 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800898f-4800-4818-b82d-4e97877c09e4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:23.297493 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a800898f-4800-4818-b82d-4e97877c09e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:23.334410 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:23.358869 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800898f-4800-4818-b82d-4e97877c09e4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:23.362993 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d1d72-c6a0-4b9e-85b4-f79cdb07effe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10d29b7934374f34923587eda9be0ed0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:23.363637 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d1d72-c6a0-4b9e-85b4-f79cdb07effe service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:23.363917 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d1d72-c6a0-4b9e-85b4-f79cdb07effe service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:23.364192 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d1d72-c6a0-4b9e-85b4-f79cdb07effe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:23.373418 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d1d72-c6a0-4b9e-85b4-f79cdb07effe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:23.374852 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d1d72-c6a0-4b9e-85b4-f79cdb07effe service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:23.385103 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1435/2887] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:40:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:40:23.909844 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748df03-73a4-4ce0-8058-b134909e4e21 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:40:23.911334 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3748df03-73a4-4ce0-8058-b134909e4e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:40:23.926094 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748df03-73a4-4ce0-8058-b134909e4e21 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:40:23.932683 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f7aa4-b65a-4e3c-b18b-7ee2858424da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06ec5a62dde8447fa4c9a00a0f1828f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1282af50d400435ba3f0e00de6e5fb67', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:40:23.933085 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f7aa4-b65a-4e3c-b18b-7ee2858424da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:40:23.933268 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f7aa4-b65a-4e3c-b18b-7ee2858424da admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:40:23.933859 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f7aa4-b65a-4e3c-b18b-7ee2858424da admin admin] PATH_INFO: `/v3/projects/80be84e153cf4739a12c043a4062ad5f` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:40:23.936127 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630f7aa4-b65a-4e3c-b18b-7ee2858424da admin admin] RBAC: Authorizing `identity:delete_project(project_id=80be84e153cf4739a12c043a4062ad5f)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:40:23.940138 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630f7aa4-b65a-4e3c-b18b-7ee2858424da admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:40:24.002735 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-630f7aa4-b65a-4e3c-b18b-7ee2858424da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80be84e153cf4739a12c043a4062ad5f', 'request_id': 'req-630f7aa4-b65a-4e3c-b18b-7ee2858424da'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:24.012139 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-630f7aa4-b65a-4e3c-b18b-7ee2858424da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80be84e153cf4739a12c043a4062ad5f', 'request_id': 'req-630f7aa4-b65a-4e3c-b18b-7ee2858424da'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:24.018833 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-630f7aa4-b65a-4e3c-b18b-7ee2858424da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80be84e153cf4739a12c043a4062ad5f', 'request_id': 'req-630f7aa4-b65a-4e3c-b18b-7ee2858424da'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:24.040455 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-630f7aa4-b65a-4e3c-b18b-7ee2858424da admin admin] The token cache is being invalidate because project 80be84e153cf4739a12c043a4062ad5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 10:40:24.040778 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-630f7aa4-b65a-4e3c-b18b-7ee2858424da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:24.041487 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-630f7aa4-b65a-4e3c-b18b-7ee2858424da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 10:40:24.043199 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1453/2888] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jul 18 10:40:23 2022] DELETE /identity/v3/projects/80be84e153cf4739a12c043a4062ad5f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 10:40:33.380837 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=11 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:40:34.030926 dv-cinder-837546-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 10:43:02.690939 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d1d72-c6a0-4b9e-85b4-f79cdb07effe service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 10:43:02.692351 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4c8d1d72-c6a0-4b9e-85b4-f79cdb07effe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:02.765407 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d1d72-c6a0-4b9e-85b4-f79cdb07effe service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 10:43:02.830552 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9a18d-a73c-4c46-8619-be3e9419b1e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd07165f554a445169b2c3f0d173c0bb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e71a3acd0adf46a2a35b965a27c4998f', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 10:43:02.831204 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9a18d-a73c-4c46-8619-be3e9419b1e1 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 10:43:02.831487 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9a18d-a73c-4c46-8619-be3e9419b1e1 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 10:43:02.831763 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9a18d-a73c-4c46-8619-be3e9419b1e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 10:43:02.833608 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2ad9a18d-a73c-4c46-8619-be3e9419b1e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 10:43:02.846234 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad9a18d-a73c-4c46-8619-be3e9419b1e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 10:43:02.847846 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad9a18d-a73c-4c46-8619-be3e9419b1e1 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 10:43:02.859589 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1436/2889] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jul 18 10:43:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 10:43:12.854712 dv-cinder-837546-3-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=58 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}